Audio processing method, audio processing apparatus, and electronic device

Abstract

The application provides an audio processing method, an audio processing device and an electronic equipment, and belongs to the technical field of audio processing. In the method, first, the audio data to be processed is subjected to frame processing to obtain a plurality of audio frames to be processed, and a feature vector of each audio frame to be processed is determined; second, the feature vector of the audio frame to be processed is subjected to clustering analysis by a classification model to obtain a target audio category to which the audio frame to be processed belongs; finally, for the audio frames to be processed of different target audio categories, an audio effect processing method matched with the target audio type is adopted for processing to obtain processed audio frames, and the plurality of processed audio frames are subjected to splicing processing, so that the processed audio data can be obtained. Through the method of the application, the noise of different audio types in the audio data to be processed can be comprehensively suppressed, so that the output processed audio data is more pleasant to listen to; meanwhile, the occupation of computing resources is reduced, and the processing efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of audio processing, and in particular to an audio processing method, an audio processing device and an electronic device. BACKGROUND

[0002] With the rapid development of electronic devices, micro loudspeakers have been widely used in portable electronic devices such as smart phones and Internet of Things terminals. However, due to the small cavity of the micro loudspeaker, when playing an audio signal with energy concentrated in low frequencies, the insufficient cavity space of the micro loudspeaker will cause a large amount of distortion and generate noise, thereby affecting the user's auditory experience.

[0003] Piano sound is a type of audio that is prone to generating noise. In order to ensure the sound quality of the audio signal, it is necessary to identify and process the piano sound in the audio signal. In related technologies, the identification and processing of piano sound usually relies on a multi-level classification algorithm. First, a neural network is used to classify the audio signal once to distinguish between human voice and background sound. Second, the audio segment classified as background sound is classified again to determine whether it is piano sound. Finally, when the secondary classification result is piano sound, the audio segment is suppressed to reduce the interference of piano sound and reduce the noise emitted by the micro loudspeaker. However, the method of identifying piano sound based on a multi-level classification algorithm requires a large amount of computing resources and high hardware performance in actual use. At the same time, other audio types may also generate noise, and the existing method only processes piano sound, resulting in a one-sided approach to noise elimination.

[0004] Therefore, in the existing audio processing method, the consumption of computing resources is large, and the noise of other audio types except piano sound cannot be effectively suppressed, which is a problem to be solved. SUMMARY

[0005] The present application provides an audio processing method, an audio processing device and an electronic device to solve the problem of large consumption of computing resources and inability to effectively suppress the noise of other audio types except piano sound in the existing audio processing method.

[0006] In a first aspect, the present application provides an audio processing method, which comprises:

[0007] frame processing on the to-be-processed audio data to obtain a plurality of to-be-processed audio frames;

[0008] determining a feature vector of any to-be-processed audio frame in the plurality of to-be-processed audio frames;

[0009] perform clustering analysis on the feature vector of the to-be-processed audio frame through a classification model to obtain a target audio category to which the to-be-processed audio frame belongs; the target audio category is one of a plurality of preset audio categories, and the plurality of preset audio categories at least include piano sound, electronic sound, and single-frequency sound;

[0010] perform sound effect processing on the to-be-processed audio frame according to a processing method matched with the target audio category based on the target audio category to which the to-be-processed audio frame belongs, to obtain a processed audio frame corresponding to the to-be-processed audio frame;

[0011] perform splicing processing on the processed audio frames corresponding to the plurality of to-be-processed audio frames respectively, to obtain processed audio data.

[0012] In a possible design, the performing clustering analysis on the feature vector of the to-be-processed audio frame through a classification model to obtain a target audio category to which the to-be-processed audio frame belongs includes:

[0013] perform clustering analysis on the feature vector of the to-be-processed audio frame through a Gaussian mixture model to obtain a first probability that the to-be-processed audio frame belongs to each of the plurality of preset audio categories; the Gaussian mixture model is obtained by training a plurality of audio frame samples;

[0014] determine, as the target audio category to which the to-be-processed audio frame belongs, the preset audio category corresponding to the maximum first probability.

[0015] In a possible design, the performing sound effect processing on the to-be-processed audio frame according to a processing method matched with the target audio category based on the target audio category to which the to-be-processed audio frame belongs, to obtain a processed audio frame corresponding to the to-be-processed audio frame, includes:

[0016] perform attenuation or enhancement processing on a signal amplitude in a frequency band range matched with the target audio category in the to-be-processed audio frame based on a preset value matched with the target audio category, to obtain the processed audio frame corresponding to the to-be-processed audio frame.

[0017] In a possible design, in a case where the target audio category is piano sound, the performing attenuation or enhancement processing on a signal amplitude in a frequency band range matched with the target audio category in the to-be-processed audio frame based on a preset value matched with the target audio category, to obtain the processed audio frame corresponding to the to-be-processed audio frame, includes:

[0018] performing attenuation processing on signal amplitudes in a third preset frequency band range of the to-be-processed audio frame based on a third preset value, to obtain the processed audio frame.

[0019] In a possible design, in a case where the target audio category is an electronic sound, performing attenuation or enhancement processing on signal amplitudes in a frequency band range of the to-be-processed audio frame that matches the target audio category based on a preset value that matches the target audio category, to obtain a processed audio frame corresponding to the to-be-processed audio frame, includes:

[0020] performing attenuation processing on signal amplitudes in a third preset frequency band range of the to-be-processed audio frame based on a third preset value, to obtain the processed audio frame.

[0021] In a possible design, in a case where the target audio category is a single-frequency sound, performing attenuation or enhancement processing on signal amplitudes in a frequency band range of the to-be-processed audio frame that matches the target audio category based on a preset value that matches the target audio category, to obtain a processed audio frame corresponding to the to-be-processed audio frame, includes:

[0022] performing attenuation processing on signal amplitudes of the to-be-processed audio frame based on a preset static gain, to obtain the processed audio frame.

[0023] In a possible design, the plurality of preset audio categories further include other sounds, and in a case where the target audio category is the other sound, performing audio effect processing on the to-be-processed audio frame according to a processing method that matches the target audio category based on the target audio category to which the to-be-processed audio frame belongs, to obtain a processed audio frame corresponding to the to-be-processed audio frame, includes:

[0024] determining the to-be-processed audio frame as the processed audio frame.

[0025] In a possible design, the frame processing on the to-be-processed audio data to obtain a plurality of to-be-processed audio frames includes:

[0026] performing frame processing on the to-be-processed audio data to obtain a plurality of first audio frames;

[0027] performing windowing processing on the plurality of first audio frames respectively to obtain a plurality of second audio frames;

[0028] performing pre-emphasis processing on the plurality of second audio frames respectively to obtain a plurality of to-be-processed audio frames.

[0029] In a second aspect, the present application provides an audio processing method and device, the device comprising:

[0030] a first processing module configured to perform frame processing on the to-be-processed audio data to obtain a plurality of to-be-processed audio frames;

[0031] a determining module configured to determine a feature vector of any of the to-be-processed audio frames;

[0032] a clustering analysis module configured to perform clustering analysis on the feature vector of the to-be-processed audio frame by using a classification model to obtain a target audio category to which the to-be-processed audio frame belongs; wherein the target audio category is one of a plurality of preset audio categories, and the plurality of preset audio categories at least include a piano sound, an electronic sound, and a single-frequency sound;

[0033] a second processing module configured to perform audio effect processing on the to-be-processed audio frame according to a processing method matched with the target audio category to which the to-be-processed audio frame belongs, to obtain a processed audio frame corresponding to the to-be-processed audio frame;

[0034] a third processing module configured to perform splicing processing on the processed audio frames corresponding to the plurality of to-be-processed audio frames respectively, to obtain processed audio data.

[0035] In a third aspect, the present application provides an electronic device comprising a memory and at least one processor.

[0036] The memory stores computer-executable instructions.

[0037] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor performs the method according to the first aspect or various possible designs of the first aspect.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed, the method according to the first aspect or various possible designs of the first aspect is implemented.

[0039] In a fifth aspect, the present application provides a computer program product, wherein the computer program product comprises computer program code, and when the computer program code is run on a computer, the computer is caused to implement the method according to the first aspect or various possible designs of the first aspect.

[0040] In a sixth aspect, the present application provides a chip, comprising: an interface circuit and a logic circuit, the interface circuit is configured to receive a signal from another chip outside the chip and transmit the signal to the logic circuit, or send a signal from the logic circuit to another chip outside the chip, and the logic circuit is configured to implement the method of the first aspect or various possible designs of the first aspect.

[0041] The embodiment of the present application provides an audio processing method, an audio processing device and an electronic device. In the method, first, frame processing is performed on to-be-processed audio data to obtain a plurality of to-be-processed audio frames, and a feature vector of each to-be-processed audio frame is determined; second, the feature vector of the to-be-processed audio frame is analyzed by a classification model to determine a target audio category to which the to-be-processed audio frame belongs, and the target audio category is a piano sound, an electronic sound or a single-frequency sound; and finally, according to the target audio category to which the to-be-processed audio frame belongs, the to-be-processed audio frame is processed according to a processing method matched with the target audio category, to obtain a processed audio frame corresponding to the to-be-processed audio frame, and the plurality of processed audio frames are spliced to obtain processed audio data. In the present application, the target audio category can be a piano sound, an electronic sound or a single-frequency sound. For the to-be-processed audio frame of different target audio categories, a sound effect processing method matched with the target audio type is used for processing, so that comprehensive suppression of different audio types of noise in the to-be-processed audio data can be realized, thereby making the output processed audio data more pleasant. Compared with the multi-level classification algorithm used in the prior art, the present application can realize multi-classification of the to-be-processed audio frame through only one classification model, not only reducing the occupation of computing resources, but also effectively improving the processing efficiency, and effectively avoiding the problem of excessive consumption of computing resources in the traditional method. BRIEF DESCRIPTION OF DRAWINGS

[0042] Figure 1 A flowchart of a piano sound recognition method provided by the prior art;

[0043] Figure 2 A flowchart of an audio processing method provided by the embodiment of the present application;

[0044] Figure 3 A flowchart of another audio processing method provided by the embodiment of the present application;

[0045] Figure 4 A flowchart of still another audio processing method provided by the embodiment of the present application;

[0046] Figure 5 A spectrum diagram of a piano sound provided by the embodiment of the present application;

[0047] Figure 6A spectrum diagram of an electronic tone provided by an embodiment of the present application;

[0048] Figure 7 A spectrum diagram of a single tone provided by an embodiment of the present application;

[0049] Figure 8 A structure diagram of an audio processing device provided by an embodiment of the present application;

[0050] Figure 9 A structure diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0052] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs; the terminology used in the specification herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the use of the terms "including," "comprising," "having" and variations thereof herein is meant to encompass the inclusion of the recited elements but not the exclusion of others not recited.

[0053] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase "an embodiment" in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily mutually exclusive of one another. A person of ordinary skill in the art will understand that the embodiments described herein can be combined with one another.

[0054] The term "and / or" in this document is used to describe a correlation relationship of associated objects, which means that there can be three relationships, for example, A and / or B can mean that A exists, A and B exist at the same time, and B exists. In addition, the character " / " in this document generally represents an "or" relationship between the front and rear associated objects.

[0055] In addition, the terms "first", "second", and the like in the specification and claims of the present application or the above drawings are used to distinguish different objects, and are not used to describe a specific order, which can explicitly or implicitly include one or more of the features.

[0056] In the description of the present application, the meaning of "a plurality of" and "at least two" is two or more (including two), and similarly, "a plurality of groups" and "at least two groups" means two or more groups (including two groups).

[0057] In the description of the present application, it should be noted that, unless otherwise specified and limited, the terms "connected", "connected" should be understood broadly, for example, "connected" or "connected" can refer to physical connection, but also can refer to electrical connection or signal connection, for example, it can be directly connected, that is, physical connection, or indirectly connected through at least one element, as long as the circuit is connected, it can also be the internal connection of two elements; Signal connection can be connected through the circuit, but also can be connected through the media medium, for example, radio waves. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0058] In order for those skilled in the art to better understand the scheme of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the drawings. It should be noted that, in the case of no conflict, different technical features in the present application can be combined with each other.

[0059] With the rapid development of electronic equipment, micro loudspeakers are widely used in portable electronic equipment such as smart phones and Internet of Things terminals. However, due to the small cavity of the micro loudspeaker, when playing audio signals with energy concentrated in low frequency, the insufficient cavity space of the micro loudspeaker will cause a large amount of distortion when the audio signal is played, producing a "rattling" sound, which is usually called noise.

[0060] Piano sound is a type of audio that is prone to noise. In order to ensure the sound quality of the audio signal, it is necessary to identify and process the piano sound.

[0061] Figure 1 A flowchart of a piano sound identification and processing method provided by the prior art is shown in FIG. 1. Figure 1As shown, since the rustling noise and the like in the music is generally caused by the piano sound with high energy, in order to avoid the mis-pressing of the human voice and the non-piano sound in the song and further affect the listening feeling of the music when processing the audio, first, the first neural network model is used to distinguish the human voice and the background sound in the input audio, if the sound signal is the human voice, the sound signal is directly output as the output audio, if the sound signal is the background sound, whether the background sound is the piano sound is identified through the second neural network model. In the case that the second neural network model identifies that the background sound is the piano sound, the piano sound is dynamically suppressed according to the amplitude of the piano sound, the suppressed background sound is obtained, and the suppressed background sound is output as the output audio; in the case that the second neural network model does not identify that the background sound is the piano sound, the background sound is directly output as the output audio, and further the identification processing of the piano sound is realized.

[0062] Although the above-mentioned identification processing method of the piano sound can identify and process the piano sound in the input audio, it needs to occupy more computing resources and has a higher requirement for hardware performance; at the same time, since other audio types can also produce noise, the existing method only identifies and processes the piano sound, and cannot process the noise of other audio types except the piano sound, resulting in that the noise elimination is one-sided.

[0063] Based on the problems in the related art, the present application provides an audio processing method, first, the audio data to be processed is frame-processed to obtain a plurality of audio frames to be processed, and the feature vector of each audio frame to be processed is determined; second, the feature vector of the audio frame to be processed is analyzed by a classification model to obtain the target audio category to which the audio frame to be processed belongs; finally, for the audio frame to be processed of different target audio categories, an audio effect processing method matched with the target audio type is used for processing to obtain a processed audio frame, and the plurality of processed audio frames are spliced to obtain the processed audio data.

[0064] In the present application, for the audio frame to be processed of different target audio categories, an audio effect processing method matched with the target audio category is used for processing. Since the target audio category can be the piano sound, the electronic sound or the single frequency sound, the present application can suppress the noise in the piano sound, the electronic sound and the single frequency sound respectively, realize the comprehensive suppression of the noise of multiple audio types in the audio data to be processed, and thus make the output processed audio data more pleasant. Compared with the multi-level classification algorithm used in the prior art, the present application can realize the multi-classification of the audio frame to be processed through only one classification model, not only reduces the occupation of computing resources, but also effectively improves the processing efficiency, and effectively avoids the problem of excessive consumption of computing resources in the traditional method.

[0065] Next, through some specific embodiments and drawings, how the application solves the problems of large consumption of computing resources and failure to effectively suppress the noise of other audio types except for the piano sound in the prior art audio processing method will be described in detail.

[0066] Figure 2 A flowchart of an audio processing method provided by an embodiment of the application is shown in FIG. 1. As shown in FIG. 1, the audio processing method provided by the embodiment of the application specifically includes S201 to S205, which will be described in detail below. Figure 2

[0067] It should be noted that the execution subject of the audio processing method provided by the embodiment of the application can be a controller in a terminal device, and the terminal device can be a mobile phone, a computer, etc.

[0068] S201, performing frame processing on to-be-processed audio data to obtain a plurality of to-be-processed audio frames.

[0069] The to-be-processed audio data is long-time audio data, and the to-be-processed audio frame is a to-be-processed audio segment corresponding to a preset time period in the to-be-processed audio data.

[0070] In this embodiment, the to-be-processed audio data is continuous signal, and the to-be-processed audio data contains various complex frequency components. When performing spectrum analysis on the to-be-processed audio data directly, the analysis process is complex, and the accuracy of the analysis result is poor. By dividing the to-be-processed audio data into a plurality of to-be-processed audio frames, spectrum analysis can be performed on each to-be-processed audio frame separately, so as to accurately extract the spectrum features of each to-be-processed audio frame and improve the accuracy of the spectrum representation of the to-be-processed audio frame. In addition, in the subsequent process, a feature vector can also be calculated and classified for each to-be-processed audio frame, avoiding global calculation of the to-be-processed audio data, so that the processing process of the to-be-processed audio data is more efficient.

[0071] S202, determining a feature vector of any to-be-processed audio frame in the plurality of to-be-processed audio frames.

[0072] The feature vector of the to-be-processed audio frame is a mathematical description of the spectrum of the to-be-processed audio frame, and is used to indicate the most representative features in the to-be-processed audio frame.

[0073] It should be noted that the feature vector of the to-be-processed audio frame can include a plurality of feature parameters of the to-be-processed audio frame. The plurality of feature parameters of the to-be-processed audio frame at least include the zero-crossing rate, the fundamental frequency, the spectrum center and the bandwidth of the to-be-processed audio frame.

[0074] ​After the feature parameters of the to-be-processed audio frame are calculated, all the feature parameters of the to-be-processed audio frame are spliced into a vector, that is, the feature vector of the to-be-processed audio frame is obtained. A multi-dimensional vector of the feature vector of the to-be-processed audio frame can represent the spectral characteristics of the to-be-processed audio frame.

[0075] In this embodiment, by calculating the feature parameters of each to-be-processed audio frame and generating the feature vector of each to-be-processed audio frame, the audio signal corresponding to the to-be-processed audio frame can be converted from time domain to mathematical representation (feature vector) of frequency domain information. The feature vector of the to-be-processed audio frame can effectively capture the key information of the audio signal corresponding to the to-be-processed audio frame, so that the subsequent classification and sound effect processing of the to-be-processed audio frame are more accurate.

[0076] S203, performing clustering analysis on the feature vector of the to-be-processed audio frame by using the classification model to obtain a target audio category to which the to-be-processed audio frame belongs.

[0077] The target audio category is one of a plurality of preset audio categories.

[0078] The plurality of preset audio categories at least include piano sound, electronic sound, and single frequency sound.

[0079] It should be noted that the piano sound refers to the sound produced by a piano. The piano sound has complex harmonic components, and the spectrum contains multiple frequency components, especially the low frequency part is relatively strong, and the amplitude of the high frequency component gradually decays. The piano sound usually has rich timbre and obvious pitch change, and belongs to natural audio.

[0080] The electronic sound refers to the sound produced by an electronic synthesizer or other electronic musical instruments. The electronic sound has a relatively regular spectrum and strong low frequency components. Compared with natural audio, the frequency distribution of the electronic sound is complex and irregular.

[0081] The single frequency sound refers to an audio signal composed of only one pure frequency component, usually in the form of a sine wave. The single frequency sound has no complex harmonic components, and the spectrum only has one single frequency peak, which is commonly used in experimental sound, benchmark test sound or signal debugging in electronic devices.

[0082] The classification model is a pre-trained model, which can identify the feature vector of the to-be-processed audio frame and predict the target audio category to which the to-be-processed audio frame belongs based on the feature vector of the to-be-processed audio frame.

[0083] It should be noted that the classification model can be a Bayesian mixture model, a Gaussian mixture model (Gaussian mixture model, GMM), or other models capable of multi-classification of the feature vector of the to-be-processed audio frame, which is not limited in the present embodiment.

[0084] In this embodiment, the feature vector is clustered and analyzed by the classification model, so that the target audio category to which the to-be-processed audio frame belongs can be accurately obtained, the sound effect processing of the to-be-processed audio frame can be more fine and effective, the audio quality can be maximized, manual intervention and a large amount of calculation are avoided, and resource consumption is reduced.

[0085] S204, based on the target audio category to which the to-be-processed audio frame belongs, performing sound effect processing on the to-be-processed audio frame according to a processing method matched with the target audio category to obtain a processed audio frame corresponding to the to-be-processed audio frame.

[0086] Among them, the audio of different preset audio categories has different characteristics, when processing the audio of different preset audio categories, different processing methods need to be used according to the audio characteristics of different preset audio categories, so as to realize the improvement of audio quality, reduce audio noise, and enhance audio effect.

[0087] It should be noted that the audio characteristics of piano sound, electronic sound and single frequency sound are different, therefore, when the target audio category to which each to-be-processed audio frame belongs is different, the sound effect processing is performed on each to-be-processed audio frame according to the processing method matched with the target audio category to which each to-be-processed audio frame belongs, to obtain a processed audio frame corresponding to each to-be-processed audio frame. The processed audio frame can better express the audio characteristics of the to-be-processed audio frame and has higher sound quality.

[0088] S205, performing splicing processing on the processed audio frames corresponding to the plurality of to-be-processed audio frames respectively to obtain processed audio data.

[0089] In a possible implementation, the plurality of processed audio frames can be directly spliced in time sequence to obtain the processed audio data corresponding to the to-be-processed audio data.

[0090] In another possible implementation, each processed audio frame can be windowed (such as Hanning window, Hamming window, etc.), to obtain a windowed audio frame, and then the plurality of windowed audio frames can be directly spliced in time sequence to obtain the processed audio data corresponding to the to-be-processed audio data.

[0091] By windowing the processed audio frame, the edge of the processed audio frame can be smoothed, effectively reducing the abruptness of the splicing point when the two adjacent processed audio frames are directly spliced, and ensuring that the volume change between the two adjacent frames in the processed audio data will not cause obvious jump or audio break.

[0092] The embodiment of the present application provides an audio processing method, first, frame processing is performed on to-be-processed audio data, a plurality of to-be-processed audio frames are obtained, and a feature vector of each to-be-processed audio frame is determined; second, the feature vector of the to-be-processed audio frame is analyzed by clustering through a classification model, so that a target audio category to which the to-be-processed audio frame belongs is determined, and the target audio category is a piano sound, an electronic sound or a single-frequency sound; finally, according to the target audio category to which the to-be-processed audio frame belongs, the to-be-processed audio frame is processed in an audio effect according to a processing method matched with the target audio category, a processed audio frame corresponding to the to-be-processed audio frame is obtained, and a plurality of processed audio frames are spliced, so that processed audio data is obtained. In the present application, the target audio category can be a piano sound, an electronic sound or a single-frequency sound, the to-be-processed audio frame of different target audio categories is processed by using an audio effect processing method matched with the target audio type, so that comprehensive suppression of different audio types of noise in the to-be-processed audio data can be realized, thereby making the output processed audio data more pleasant. Compared with the multi-level classification algorithm used in the prior art, the present application can realize multi-classification of the to-be-processed audio frame through only one classification model, not only reduces the occupation of computing resources, but also effectively improves the processing efficiency, and effectively avoids the problem of excessive consumption of computing resources in the traditional method.

[0093] In the above embodiment, for any to-be-processed audio frame in the plurality of to-be-processed audio frames, the terminal device needs to determine the feature vector of any to-be-processed audio frame in the plurality of to-be-processed audio frames. Next, the specific process that the terminal device determines the feature vector of any to-be-processed audio frame in the plurality of to-be-processed audio frames is described in detail.

[0094] Figure 3 Another flowchart of an audio processing method provided by the embodiment of the present application is shown in FIG. 6. As shown in FIG. 6, in a possible embodiment, S202 can be implemented through S2021 to S2023, and the following describes S2021 to S2023 in detail. Figure 3

[0095] S2021, performing time-frequency transformation on the plurality of to-be-processed audio frames respectively to obtain the spectral representation of the plurality of to-be-processed audio frames.

[0096] In the embodiment, Fourier transform or short-time Fourier transform and the like can be used to perform time-frequency transformation on the to-be-processed audio frame to obtain the spectral representation of the to-be-processed audio frame, that is, the distribution of each frequency component of the to-be-processed audio frame in time, which provides important frequency information for subsequent feature parameter calculation of the to-be-processed audio frame.

[0097] S2022, determining the zero-crossing rate, the fundamental frequency, the spectral center and the bandwidth of the to-be-processed audio frame according to the spectral representation of the to-be-processed audio frame.

[0098] ​The zero-crossing rate (ZCR) of the audio frame being processed refers to the number of times the signal of the audio frame transitions from a positive value to a negative value or from a negative value to a positive value. The zero-crossing rate of the audio frame being processed is used to measure the "coarseness" or frequency of change of the signal of the audio frame being processed.

[0099] Specifically, the zero-crossing rate is used to measure the frequency of signal symbol changes in an audio frame being processed. Generally, a change in a signal from a positive value to a negative value or vice versa is called a "zero-crossing," and the zero-crossing rate is the number of zero-crossing points per unit time. A higher zero-crossing rate usually indicates rapid signal fluctuations, such as noise or complex signals; a lower zero-crossing rate represents a more stable signal, typically a low-frequency or pure tone signal.

[0100] In this embodiment, the zero-crossing rate ZCR of the audio frame to be processed can be calculated using formula (1).

[0101] Formula (1)

[0102] In formula (1), This indicates the total number of data sampling points included in the audio frame to be processed; It is a symbolic function; Indicates the sequence number of the data sampling point. ; Indicates the first The signal value of each data sampling point; Indicates the first The signal value of each data sampling point.

[0103] The fundamental frequency of the audio frame being processed refers to the lowest frequency component within that frame. It determines the pitch of the audio signal and is the fundamental frequency component of the frame. The fundamental frequency is typically determined by the vibration frequency of the sound source and is one of the important characteristics for analyzing audio signals. For natural audio signals (such as speech and instrument sounds), the fundamental frequency is the primary basis for determining the pitch of the audio signal.

[0104] Specifically, the fundamental frequency of the audio frame to be processed can be obtained by calculating the autocorrelation function of the audio signal of the audio frame to be processed.

[0105] The autocorrelation function of the audio signal of the audio frame to be processed reflects the similarity between the audio signal of the audio frame to be processed and its delayed version. By analyzing the autocorrelation function of the audio signal of the audio frame to be processed, the periodicity of the audio signal of the audio frame to be processed can be determined, thereby inferring the fundamental frequency of the audio frame to be processed.

[0106] The fundamental frequency is typically calculated based on the delay time corresponding to the maximum value of the autocorrelation function. Specifically, for , Short-time autocorrelation function The short-time autocorrelation function can be calculated by formula (2).

[0107] Formula (2)

[0108] In formula (2), represents the audio signal obtained at time delay ; represents the correlation of and . By calculating the value of the autocorrelation function , the corresponding to the maximum is found, that is, the fundamental frequency of the audio frame to be processed can be calculated.

[0109] Among them, the spectral center of the audio frame to be processed is the "center of gravity" of the spectrum of the audio signal of the audio frame to be processed, which is used to describe the concentrated position of the frequency distribution of the audio signal. The spectral center can reflect the timbre characteristics of the audio signal, and is related to the brightness or clarity of the audio signal. Generally speaking, the audio signal with higher spectral center will appear brighter, and lower spectral center may exhibit a more turbid sound quality.

[0110] Specifically, the spectral center is obtained by calculating the weighted average position of each frequency component in the spectrum, wherein the weight of the frequency component is determined by the energy size of each frequency. The spectral center of the audio frame to be processed can be calculated by formula (3).

[0111] Formula (3)

[0112] In formula (3), is a frequency variable in the audio frame to be processed; is the spectrum of the audio signal with the frequency variable ; is the spectral energy with the frequency variable ; is the largest frequency component in the audio frame to be processed.

[0113] Among them, the bandwidth (Bandwidth, BW) of the audio frame to be processed refers to the width of the spectrum of the audio signal of the audio frame to be processed, that is, the range of effective frequency components in the audio signal of the processing audio frame. The bandwidth of the audio frame to be processed reflects the width of the frequency distribution of the audio signal of the audio frame to be processed.

[0114] ​It should be noted that, for a speech signal, the bandwidth is usually concentrated between 0.3 kHz and 3.4 kHz, while the bandwidth of a music signal is usually wider than that of a speech signal, and the bandwidth of a music signal is about 22.05 kHz, which can cover a wider frequency range.

[0115] Specifically, the bandwidth of the audio frame to be processed The bandwidth of the audio frame to be processed can be calculated by formula (4).

[0116] Formula (4)

[0117] S2023, based on the zero-crossing rate, the fundamental frequency, the spectral center and the bandwidth of the audio frame to be processed, obtaining the feature vector of the audio frame to be processed.

[0118] The feature vector of the audio frame to be processed is a group of numerical values obtained after feature extraction of the audio signal of the audio frame to be processed. The feature vector of the audio frame to be processed is composed of the zero-crossing rate, the fundamental frequency, the spectral center and the bandwidth of the audio frame to be processed, and is used to describe the key attributes of the audio signal of the audio frame to be processed.

[0119] It should be noted that the feature vector of the audio frame to be processed is the basis for subsequent classification of the audio frame to be processed. The feature vector of the audio frame to be processed converts the complex audio signal into a numerical form that is easy to analyze, and can provide input for the classification model.

[0120] Specifically, the zero-crossing rate, the fundamental frequency, the spectral center and the bandwidth of the audio frame to be processed are spliced, that is, the feature vector of the audio frame to be processed is obtained.

[0121] For example, the zero-crossing rate of the audio frame to be processed is 0.8, the fundamental frequency is 300 Hz, the spectral center is 1500 Hz, and the bandwidth is 500 Hz. The feature vector of the audio frame to be processed can be [0.8, 300, 1500, 500].

[0122] In the embodiments of the present application, the zero-crossing rate, the fundamental frequency, the spectral center and the bandwidth of the audio frame to be processed are calculated, and the calculated zero-crossing rate, the fundamental frequency, the spectral center and the bandwidth of the audio frame to be processed are spliced, that is, the feature vector of the audio frame to be processed is obtained. The feature vector of the audio frame to be processed can accurately describe the frequency characteristics of the audio signal of the audio frame to be processed, and provide a key basis for subsequent classification of the audio frame to be processed, effectively improving the accuracy of the classification model in classifying the audio frame to be processed.

[0123] In the above embodiment, the terminal device needs to perform clustering analysis on the feature vector of the to-be-processed audio frame through the classification model to obtain the target audio category to which the to-be-processed audio frame belongs. Next, the specific process of the terminal device performing clustering analysis on the feature vector of the to-be-processed audio frame through the classification model to obtain the target audio category to which the to-be-processed audio frame belongs is described in detail.

[0124] Figure 3 Another flowchart of an audio processing method provided in the embodiment of the present application is shown in FIG. 3. As shown in FIG. 3, in a possible embodiment, S203 can be implemented through S2031 and S2032, and the following describes S2031 and S2032 in detail. Figure 3

[0125] S2031, performing clustering analysis on the feature vector of the to-be-processed audio frame through a Gaussian mixture model to obtain a first probability that the to-be-processed audio frame respectively belongs to a plurality of preset audio categories.

[0126] The Gaussian mixture model is obtained by training a plurality of audio frame samples.

[0127] It should be noted that the Gaussian mixture model is a commonly used clustering algorithm based on probability. In the case where the plurality of audio frame samples belong to the preset audio categories, the Gaussian mixture model is trained through the plurality of audio frame samples, the Gaussian mixture model learns the features of the plurality of audio frame samples, can obtain the probability distribution of each preset audio category, and calculates the probability that the to-be-processed audio frame belongs to different preset audio categories based on the probability distribution.

[0128] When the Gaussian mixture model is trained through the plurality of audio frame samples, the training process of the Gaussian mixture model usually depends on the expectation maximization algorithm, and the parameters are continuously optimized to maximize the likelihood probability of the data points under the Gaussian mixture model to obtain the trained Gaussian mixture model.

[0129] In the embodiment, the clustering analysis on the feature vector of each to-be-processed audio frame through the Gaussian mixture model can obtain the first probability that the to-be-processed audio frame belongs to each preset audio category, i.e., the possibility that the to-be-processed audio frame belongs to each preset audio category.

[0130] S2032, determining the preset audio category corresponding to the maximum first probability as the target audio category to which the to-be-processed audio frame belongs.

[0131] It should be noted that for each to-be-processed audio frame, the Gaussian mixture model outputs the first probability that the to-be-processed audio frame respectively belongs to a plurality of preset audio categories, and selects the preset audio category corresponding to the maximum first probability as the target audio category to which the to-be-processed audio frame belongs.

[0132] ​For example, the probability of the to-be-processed audio frame belonging to the piano sound is 0.7, the probability of the to-be-processed audio frame belonging to the electronic sound is 0.2, and the probability of the to-be-processed audio frame belonging to the single-frequency sound is 0.1, which are output by the Gaussian mixture model; in this case, the target audio category to which the to-be-processed audio frame belongs is the piano sound.

[0133] For another example, the probability of the to-be-processed audio frame belonging to the piano sound is 0.1, the probability of the to-be-processed audio frame belonging to the electronic sound is 0.6, and the probability of the to-be-processed audio frame belonging to the single-frequency sound is 0.3, which are output by the Gaussian mixture model; in this case, the target audio category to which the to-be-processed audio frame belongs is the electronic sound.

[0134] In the embodiments of the present application, by performing clustering analysis on the to-be-processed audio frame through the Gaussian mixture model, the probability of the to-be-processed audio frame belonging to multiple preset audio categories can be predicted based on the feature vector of the to-be-processed audio frame, and the target audio category to which the to-be-processed audio frame belongs can be determined according to the maximum probability value, thereby effectively improving the accuracy and robustness when classifying the to-be-processed audio frame.

[0135] In the above embodiments, the terminal device needs to perform audio effect processing on the to-be-processed audio frame according to the processing method matched with the target audio category based on the target audio category to which the to-be-processed audio frame belongs, to obtain the processed audio frame corresponding to the to-be-processed audio frame. Next, the specific process of the terminal device performing audio effect processing on the to-be-processed audio frame according to the processing method matched with the target audio category to obtain the processed audio frame corresponding to the to-be-processed audio frame is described in detail.

[0136] Figure 4 A flowchart of another audio processing method provided by the embodiments of the present application is shown in FIG. 6. As shown in FIG. 6, in a possible embodiment, the method step shown in S204 can be implemented by S2041, which is described in detail below. Figure 4

[0137] S2041, attenuating or enhancing the signal amplitude in the frequency band range matched with the target audio category in the to-be-processed audio frame based on the preset value matched with the target audio category, to obtain the processed audio frame corresponding to the to-be-processed audio frame.

[0138] It should be noted that the preset value matched with the target audio category is set by the user according to the audio characteristics of the audio corresponding to the target audio category, and the embodiments of the present application do not make specific limitations thereto.

[0139] The frequency band range matched with the target audio category is also set by the user according to the audio characteristics of the audio corresponding to the target audio category, and the embodiments of the present application do not make specific limitations thereto.

[0140] ​Specifically, in a possible embodiment, when the target audio category to which the audio frame to be processed belongs is piano sound, S2041 can be implemented by Sa, which is described in detail below.

[0141] Sa, attenuates the signal amplitude in the first preset frequency band range of the audio frame to be processed based on a first preset value, and enhances the signal amplitude in the second preset frequency band range of the audio frame to be processed based on a second preset value, to obtain the processed audio frame.

[0142] The first preset frequency band range is smaller than the second preset frequency band range.

[0143] The first preset frequency band range is a frequency band range in which piano sound energy is concentrated.

[0144] It should be noted that, since piano sound is relatively bright in subjective listening, when the target audio category to which the audio frame to be processed belongs is piano sound, attenuating the signal amplitude in the first preset frequency band range of the audio frame to be processed based on the first preset value can effectively reduce unnecessary high-frequency noise, while avoiding blurring of the sound quality.

[0145] The first preset value and the first preset frequency band range can be set by the user, and the present embodiment does not make specific limitations thereon. For example, the first preset value is -10 dB, and the first preset frequency band range is 500 Hz to 600 Hz.

[0146] The second preset frequency band range is a high-frequency band of piano sound.

[0147] It should be noted that, by enhancing the signal amplitude in the second preset frequency band range, the loudness loss caused by attenuating the signal amplitude in the first preset frequency band range can be compensated, and the subjective listening of piano sound can be made brighter.

[0148] The second preset value and the second preset range can be set by the user, and the present embodiment does not make specific limitations thereon. For example, the second preset value is +5 dB, and the second preset frequency band range is above 2 kHz.

[0149] The terminal device can attenuate the signal amplitude in the first preset frequency band range of the audio frame to be processed by an equalizer (EQ), and enhance the signal amplitude in the second preset frequency band range of the audio frame to be processed, to obtain the processed audio frame corresponding to the audio frame to be processed.

[0150] An exemplary piano sound spectrum diagram is provided for the embodiments of the present application. As shown in FIG. 1, the horizontal axis represents the frequency, and the vertical axis represents the amplitude. Figure 5 An exemplary piano sound spectrum diagram is provided for the embodiments of the present application. As shown in FIG. 1, the horizontal axis represents the frequency, and the vertical axis represents the amplitude. Figure 5As shown, line 1 represents the spectrum of the to-be-processed audio frame whose target audio category is piano sound, and line 2 represents the spectrum of the processed audio frame obtained after performing the sound effect processing on the to-be-processed audio frame. By comparing line 1 and line 2 in FIG. 8, it can be known that the processed audio frame is obtained by attenuating the amplitude corresponding to the frequency in the range of 500 Hz to 600 Hz (the first preset frequency band range) in the to-be-processed audio frame and enhancing the amplitude corresponding to the frequency above 2 kHz (the second preset frequency band range) in the to-be-processed audio frame. Figure 5

[0151] In the embodiment of the present application, in the case where the target audio category to which the to-be-processed audio frame belongs is piano sound, different frequency bands of the audio signal of the to-be-processed audio frame are respectively attenuated and enhanced, which can effectively optimize the quality of the to-be-processed audio frame, so that the audio signal of the to-be-processed audio frame is brighter and clearer in subjective listening. The method of the embodiment of the present application can improve the sound quality of piano sound and enhance the expressiveness of piano sound, especially in the application scenario of noise reduction and high-frequency enhancement.

[0152] In the above embodiment, the terminal device needs to perform sound effect processing on the to-be-processed audio frame based on the target audio category to which the to-be-processed audio frame belongs, to obtain the processed audio frame corresponding to the to-be-processed audio frame. Next, the specific process of performing sound effect processing on the to-be-processed audio frame by the terminal device to obtain the processed audio frame corresponding to the to-be-processed audio frame when the target audio category to which the to-be-processed audio frame belongs is electronic sound is described in detail.

[0153] In a possible embodiment, in the case where the target audio category to which the to-be-processed audio frame belongs is electronic sound, S2041 can be implemented by Sc, which is described in detail below.

[0154] Sc, attenuates the signal amplitude in the third preset frequency band range of the to-be-processed audio frame based on the third preset value, to obtain the processed audio frame.

[0155] The third preset frequency band range is a frequency band range in which the energy of the electronic sound is concentrated.

[0156] It should be noted that, since the electronic sound does not have the sound characteristics of the piano sound, when performing sound effect processing on the electronic sound, only the signal amplitude in the frequency band range (the third preset frequency band range) in which the energy of the electronic sound is concentrated needs to be attenuated.

[0157] The third preset value and the third preset frequency band range can be set by the user, and the embodiment does not make specific limitations thereon. For example, the third preset value is -3 dB, and the first preset frequency band range is 500 Hz to 600 Hz.

[0158] ​The terminal device can attenuate the signal amplitude in the third preset frequency band range of the to-be-processed audio frame through a peak filter to obtain a processed audio frame corresponding to the to-be-processed audio frame.

[0159] Exemplarily, Figure 6 A spectrum diagram of electronic sound is provided for an embodiment of the present application. As Figure 6 shown, line 1 represents the spectrum of the to-be-processed audio frame of the target audio category of electronic sound, and line 2 represents the spectrum of the processed audio frame obtained after the to-be-processed audio frame is subjected to sound effect processing. By comparing line 1 and line 2 in Figure 6 , it can be known that the processed audio frame is obtained by attenuating the amplitude corresponding to the frequency in the range of 500 Hz to 600 Hz (third preset frequency band range) in the to-be-processed audio frame.

[0160] In the embodiment of the present application, in the case that the target audio category to which the to-be-processed audio frame belongs is electronic sound, the signal amplitude in the frequency band range in which the electronic sound energy is concentrated in the to-be-processed audio frame is subjected to attenuation processing, which can reduce the noise or unnecessary high-frequency components in the electronic sound, avoid the influence on other frequency components, and ensure the intelligibility and naturalness of the audio signal of the to-be-processed audio frame.

[0161] In the above embodiment, the terminal device needs to perform sound effect processing on the to-be-processed audio frame based on the target audio category to which the to-be-processed audio frame belongs to obtain a processed audio frame corresponding to the to-be-processed audio frame. Next, the specific process of the terminal device performing sound effect processing on the to-be-processed audio frame to obtain a processed audio frame corresponding to the to-be-processed audio frame when the target audio category to which the to-be-processed audio frame belongs is single-frequency sound is described in detail.

[0162] In a possible embodiment, in the case that the target audio category to which the to-be-processed audio frame belongs is single-frequency sound, S2041 can be implemented by Sd, and Sd is described in detail as follows.

[0163] Sd, attenuating the signal amplitude of the to-be-processed audio frame based on a preset static gain to obtain a processed audio frame.

[0164] It should be noted that single-frequency sound only contains audio signals of a single frequency component, such as pure tone, tuning fork sound, etc., and has no complex timbre variation. Therefore, when the to-be-processed audio frame is classified as single-frequency sound, the signal amplitude in the full frequency band range of the to-be-processed audio frame is attenuated based on a preset static gain to control the volume or intensity of the audio signal of the to-be-processed audio frame, so as to avoid the influence of the excessively strong single frequency component on the overall audio quality of the audio signal.

[0165] The preset static gain can be set by the user, and this embodiment does not impose specific limitations on it. For example, the preset static gain is -2dB.

[0166] After attenuating the signal amplitude of the audio frame to be processed based on a preset static gain, the resulting audio signal is the processed audio frame corresponding to the original audio frame. Compared to the original audio frame, the amplitude of the processed audio frame has been uniformly attenuated, and the energy across the entire frequency band is more balanced, resulting in improved sound quality and avoiding distortion of the audio signal caused by an overemphasis on a single frequency component.

[0167] For example, Figure 7 This is a schematic diagram of the spectrum of a single-frequency tone provided for an embodiment of this application. For example... Figure 7 As shown, line 1 represents the spectrum of the audio frame to be processed, and line 2 represents the spectrum of the processed audio frame after sound effects processing. (Comparison) Figure 7 As can be seen from lines 1 and 2, the processed audio frame is obtained by attenuating the amplitude of the frequencies in the entire frequency band of the audio frame to be processed based on a preset static gain (-2dB).

[0168] In this embodiment, when the target audio category of the audio frame to be processed is a single-frequency tone, the processed audio frame can be obtained by attenuating the signal amplitude of the audio frame to be processed based on a preset static gain. By applying a preset static gain to attenuate the amplitude of the audio frame to be processed, the naturalness of the audio signal of the audio frame to be processed can be maintained, while removing unnecessary high-amplitude components, thus avoiding unnecessary abrupt changes or interference in the audio signal of the audio frame to be processed.

[0169] In addition, several preset audio categories include other sounds. Other sounds refer to audio types other than piano sounds, electronic sounds, and single-frequency sounds.

[0170] In one possible embodiment, if the target audio category to which the audio frame to be processed belongs is another sound, S204 can be implemented by Se, which will be described in detail below.

[0171] Se, determine the audio frame to be processed as the processed audio frame.

[0172] It should be noted that when the target audio category of the audio frame to be processed is a single-frequency tone, it indicates that the characteristics of the audio frame to be processed are significantly different from piano sounds, electronic sounds, or single-frequency tones. In this case, no further complex sound effects processing is applied to the audio frame to be processed; instead, the audio frame to be processed is directly identified as the processed audio frame.

[0173] In the embodiment of the present application, when the target audio category to which the to-be-processed audio frame belongs is other sound, the to-be-processed audio frame is not subjected to complex sound effect processing, and the original characteristics of the to-be-processed audio frame are directly retained, unnecessary processing interference is avoided, and the naturalness and integrity of the audio signal of the processed audio frame are ensured.

[0174] In the above embodiment, the terminal device needs to perform frame processing on the to-be-processed audio data to obtain a plurality of to-be-processed audio frames. Next, the specific process of performing frame processing on the to-be-processed audio data by the terminal device to obtain a plurality of to-be-processed audio frames is described in detail.

[0175] In a possible embodiment, S201 can be implemented by S2011 to S2013, and S2011 to S2013 are described in detail below.

[0176] S2011, performing frame processing on the to-be-processed audio data to obtain a plurality of first audio frames.

[0177] It should be noted that the frame processing is a basic step in audio signal preprocessing. The purpose of performing frame processing on the to-be-processed audio data is to cut the continuous to-be-processed audio data into a plurality of small time windows (first audio frames), and the signal in each first audio frame has a certain continuity in time.

[0178] It should be noted that by performing frame processing on the to-be-processed audio data, the non-stationarity of the to-be-processed audio data can be effectively reduced, so that the signal of each first audio frame can be approximately regarded as a stationary signal, thereby facilitating subsequent processing and analysis.

[0179] Specifically, when performing frame processing on the to-be-processed audio data, the length of each first audio frame and the overlap degree between adjacent two first audio frames can be set by the user, and the embodiment does not make specific limitation on this.

[0180] For example, the length of the first audio frame is 40 ms, and the overlap degree between adjacent two first audio frames is 20%.

[0181] S2012, performing windowing processing on the plurality of first audio frames respectively to obtain a plurality of second audio frames.

[0182] It should be noted that since the first audio frame is extracted from a limited time window, there is discontinuity at the beginning and end of the signal of the first audio frame. By performing windowing processing on the first audio frame respectively, the influence of the discontinuity on the spectrum analysis can be effectively reduced.

[0183] In the windowing processing of the plurality of first audio frames respectively, the window function applied to each first audio frame can be a Hamming window, a Hanning window, a rectangular window, etc. The window function can smooth the boundary of the first audio frame, so that the beginning and ending part of the first audio frame gradually transitions to zero, thereby reducing spectral leakage. At the same time, the second audio frame obtained by windowing processing of the first audio frame can provide clearer frequency components in frequency domain analysis.

[0184] S2013, pre-emphasis processing is performed on the plurality of second audio frames respectively to obtain a plurality of to-be-processed audio frames.

[0185] It should be noted that the pre-emphasis processing is used to improve the energy of the high frequency part of the signal. By performing pre-emphasis processing on each second audio frame, the high frequency components in the second audio frame can be enhanced, the frequency distribution of the audio signal of the second audio frame can be balanced, the audibility of the high frequency part of the second audio frame can be improved, and the influence of the low frequency signal in the second audio frame can be effectively reduced.

[0186] The method for performing pre-emphasis processing on the second audio frames respectively to obtain the to-be-processed audio frames corresponding to the second audio frames is a prior art, and this embodiment will not be repeated here.

[0187] In the embodiments of the present application, first, the to-be-processed audio data is subjected to frame processing to obtain a plurality of first audio frames, so that the to-be-processed audio data can better adapt to spectrum analysis and reduce edge effects; second, windowing processing is performed on the plurality of first audio frames respectively to obtain a plurality of second audio frames, so that spectral leakage can be reduced and the accuracy of frequency analysis can be improved; then, pre-emphasis processing is performed on the plurality of second audio frames respectively to obtain a plurality of to-be-processed audio frames, so that the high frequency components of the audio signal can be effectively enhanced, which helps to highlight the details of the high frequency components of the audio signal.

[0188] Figure 8 A structural schematic diagram of an audio processing device provided in the embodiments of the present application is shown in FIG. 8. As shown in FIG. 8, the audio processing device 800 provided in the embodiments of the present application includes a first processing module 801, a determination module 802, a cluster analysis module 803, a second processing module 804, and a third processing module 805. Figure 8

[0189] The first processing module 801 is configured to perform frame processing on to-be-processed audio data to obtain a plurality of to-be-processed audio frames.

[0190] The determination module 802 is configured to determine a feature vector of any to-be-processed audio frame in the plurality of to-be-processed audio frames.

[0191] ​The clustering analysis module 803 is used to perform clustering analysis on the feature vectors of the audio frame to be processed through a classification model to obtain the target audio category to which the audio frame to be processed belongs; wherein, the target audio category is one of a number of preset audio categories, and the number of preset audio categories includes at least piano sound, electronic sound, and single-frequency sound.

[0192] The second processing module 804 is used to perform sound effect processing on the audio frame to be processed according to the target audio category to which the audio frame to be processed belongs, and to obtain the processed audio frame corresponding to the audio frame to be processed.

[0193] The third processing module 805 is used to splice the processed audio frames corresponding to multiple audio frames to be processed to obtain processed audio data.

[0194] This application provides an audio processing apparatus. In this apparatus, firstly, a first processing module performs frame-segmentation on the audio data to be processed, obtaining multiple audio frames to be processed, and a determination module determines the feature vector of each audio frame. Secondly, a clustering analysis module performs clustering analysis on the feature vectors of the audio frames to be processed using a classification model to determine the target audio category to which the audio frames belong. The target audio category can be piano sound, electronic sound, or single-frequency sound. Finally, a second processing module performs sound effect processing on the audio frames to be processed according to the target audio category, using a processing method matching the target audio category, to obtain the processed audio frames. A third processing module then concatenates the multiple processed audio frames to obtain the processed audio data. In this application, the target audio category can be piano sound, electronic sound, or single-frequency sound. By using a sound effect processing method matching the target audio type for audio frames to be processed in different target audio categories, comprehensive suppression of noise of different audio types in the audio data to be processed can be achieved, resulting in more pleasing output processed audio data. Compared with the multi-level classification algorithms used in existing technologies, this application can achieve multi-classification of audio frames to be processed using only one classification model, which not only reduces the consumption of computing resources, but also effectively improves processing efficiency and effectively avoids the problem of excessive consumption of computing resources in traditional methods.

[0195] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 9 As shown, the electronic device 900 provided in this embodiment includes a memory 901 and a processor 902.

[0196] The memory 901 can be a separate physical unit, and the processor 902 can be connected through the bus 903. The memory 901 and the processor 902 can also be integrated together, implemented by hardware, etc. The memory 901 is used to store program instructions, and the processor 902 invokes the program instructions to perform the operations of the terminal device in any of the above method embodiments.

[0197] Alternatively, when part or all of the methods in the above embodiments are implemented by software, the electronic device 900 can only include the processor 902. The memory 901 for storing the program is located outside the electronic device 900, and the processor 902 is connected with the memory through a circuit / wire for reading and executing the program stored in the memory. The processor 902 can be a central processing unit (CPU), a network processor (NP), or a combination of the CPU and the NP. The processor 902 can further include a hardware chip. The hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0198] The memory 901 can include volatile memory, such as random-access memory (RAM); the memory can also include non-volatile memory, such as flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory can also include a combination of the above types of memory.

[0199] Exemplarily, the present application provides a chip, comprising: an interface circuit and a logic circuit, the interface circuit is used to receive signals from other chips outside the chip and transmit to the logic circuit, or send signals from the logic circuit to other chips outside the chip, the logic circuit is used to perform the operations of the terminal device in the above method embodiments.

[0200] Exemplarily, the present application provides a computer readable storage medium having stored thereon computer program instructions, which, when executed by a processor of an electronic device, cause the electronic device to perform operations performed by the terminal device in the above method embodiments.

[0201] Exemplarily, the present application provides a computer program product, which, when executed on an electronic device, causes the electronic device to perform operations performed by the terminal device in the above method embodiments.

[0202] The above description is merely illustrative of the principles of the application and various modifications can be made by those skilled in the art. The described embodiments have been presented for purposes of illustration and description only. They are not intended to be exhaustive or to limit the application to the precise forms disclosed, and various modifications, changes, and variations apparent to those skilled in the art can be made therein without departing from the spirit and scope of the application. The scope of the application should be determined from the appended claims and their legal equivalents rather than from the exhaustive examples given.

Claims

1. An audio processing method, characterized in that, The method is applied to a miniature loudspeaker, and the method includes: The audio data to be processed is divided into frames to obtain multiple audio frames to be processed. Determine the feature vector of any one of the plurality of audio frames to be processed; The feature vectors of the audio frame to be processed are clustered using a classification model to obtain the target audio category to which the audio frame to be processed belongs; wherein, the target audio category is one of a plurality of preset audio categories, and the plurality of preset audio categories include at least piano sound, electronic sound, and single-frequency sound; Based on the target audio category to which the audio frame to be processed belongs, the audio frame to be processed is subjected to sound effect processing according to the processing method that matches the target audio category, so as to obtain the processed audio frame corresponding to the audio frame to be processed. The processed audio frames corresponding to the plurality of audio frames to be processed are spliced ​​together to obtain processed audio data. The step of performing sound effect processing on the audio frame to be processed according to the target audio category to which the audio frame to be processed belongs, and obtaining the processed audio frame corresponding to the audio frame to be processed, includes: Based on a preset value that matches the target audio category, the signal amplitude within the frequency band range that matches the target audio category of the audio frame to be processed is attenuated or enhanced to obtain the processed audio frame corresponding to the audio frame to be processed. When the target audio category is piano sound, the process of attenuating or enhancing the signal amplitude within the frequency band matching the target audio category, based on a preset value that matches the target audio category, to obtain the processed audio frame corresponding to the audio frame to be processed, includes: The signal amplitude of the audio frame to be processed is attenuated within a first preset frequency band based on a first preset value, and the signal amplitude of the audio frame to be processed within a second preset frequency band based on a second preset value, to obtain the processed audio frame; wherein, the first preset frequency band is smaller than the second preset frequency band. When the target audio category is electronic sound, the step of attenuating or enhancing the signal amplitude within the frequency band range matching the target audio category, based on a preset value that matches the target audio category, to obtain the processed audio frame corresponding to the audio frame to be processed, includes: Based on a third preset value, the signal amplitude within a third preset frequency band of the audio frame to be processed is attenuated to obtain the processed audio frame. When the target audio category is a single-frequency tone, the process of attenuating or enhancing the signal amplitude within the frequency band matching the target audio category, based on a preset value that matches the target audio category, to obtain the processed audio frame corresponding to the audio frame to be processed, includes: The signal amplitude of the audio frame to be processed is attenuated based on a preset static gain to obtain the processed audio frame.

2. The method according to claim 1, characterized in that, The step of performing cluster analysis on the feature vectors of the audio frame to be processed using a classification model to obtain the target audio category to which the audio frame to be processed belongs includes: Cluster analysis is performed on the feature vectors of the audio frames to be processed using a Gaussian mixture model to obtain the first probability that the audio frames to be processed belong to the multiple preset audio categories; wherein, the Gaussian mixture model is trained using multiple audio frame samples; The preset audio category corresponding to the highest first probability is determined as the target audio category to which the audio frame to be processed belongs.

3. The method according to claim 1, characterized in that, The plurality of preset audio categories also include other sounds. When the target audio category is one of these other sounds, the audio frame to be processed is subjected to sound effect processing based on the target audio category to which it belongs, according to a processing method matching the target audio category, to obtain the processed audio frame corresponding to the audio frame to be processed, including: The audio frame to be processed is determined as the processed audio frame.

4. The method according to claim 1, characterized in that, The process of segmenting the audio data to be processed into frames yields multiple audio frames to be processed, including: The audio data to be processed is divided into frames to obtain multiple first audio frames; Windowing is applied to each of the multiple first audio frames to obtain multiple second audio frames; The multiple second audio frames are pre-emphasized to obtain multiple audio frames to be processed.

5. An audio processing device, characterized in that, The device is used in a miniature loudspeaker, and the device includes: The first processing module is used to perform frame-segmentation processing on the audio data to be processed, resulting in multiple audio frames to be processed. The determining module is used to determine the feature vector of any one of the plurality of audio frames to be processed; The clustering analysis module is used to perform clustering analysis on the feature vector of the audio frame to be processed using a classification model to obtain the target audio category to which the audio frame to be processed belongs; wherein, the target audio category is one of a plurality of preset audio categories, and the plurality of preset audio categories include at least piano sound, electronic sound, and single-frequency sound; The second processing module is used to perform sound effect processing on the audio frame to be processed according to the target audio category to which the audio frame to be processed belongs, and to obtain the processed audio frame corresponding to the audio frame to be processed. The third processing module is used to splice the processed audio frames corresponding to the plurality of audio frames to be processed to obtain processed audio data. The second processing module is specifically used to attenuate or enhance the signal amplitude of the audio frame to be processed within the frequency band range that matches the target audio category, based on a preset value that matches the target audio category, so as to obtain the processed audio frame corresponding to the audio frame to be processed. When the target audio category is piano sound, the second processing module is specifically used to attenuate the signal amplitude within a first preset frequency band of the audio frame to be processed based on a first preset value, and to enhance the signal amplitude within a second preset frequency band of the audio frame to be processed based on a second preset value, to obtain the processed audio frame; wherein, the first preset frequency band is smaller than the second preset frequency band. When the target audio category is electronic sound, the second processing module is specifically used to attenuate the signal amplitude within a third preset frequency band of the audio frame to be processed based on a third preset value to obtain the processed audio frame. When the target audio category is a single-frequency tone, the second processing module is specifically used to attenuate the signal amplitude of the audio frame to be processed based on a preset static gain to obtain the processed audio frame.

6. An electronic device, characterized in that, include: Memory and at least one processor; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the method as described in any one of claims 1 to 4.

Images