Audio data processing method, electronic equipment, storage medium and computer program product

Abstract

The invention provides an audio data processing method, electronic equipment, a storage medium and a computer program product, and the method comprises the steps: extracting audio feature information from target audio data according to a feature extraction strategy corresponding to a specified classification algorithm; classifying the audio feature information according to the specified classification algorithm to obtain a music type of the target audio data; calling a sound effect enhancement strategy matched with the music type, and performing sound effect enhancement processing on the target audio data; wherein the sound effect enhancement strategy refers to an adjustment mode of an equalizer, a reverberation mode and a bass enhancement function. According to the scheme, classification and sound effect enhancement of the audio data are realized under the condition of relatively low hardware requirements and resource consumption.

Description

Technical Field

[0001] The present application relates to the field of audio processing technology, and in particular to an audio data processing method and electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of acoustic technology, people's requirements for music listening are gradually increasing. In audio systems such as speakers, headphones, and home theaters, the music playback effect is often difficult to meet user needs due to the influence of hardware settings, environmental acoustic effects, and personal listening habits of users. Users have different preferences for different types of music. For example: for pop music, it is necessary to emphasize the clarity of human voices and the harmony of instruments; for rock music, it is necessary to highlight the impact and power of instruments; for classical music, the sound must be accurately restored to show the layering and dynamic range of the music.

[0003] In order to meet users' listening needs for different types of music, relevant solutions generally use music classification and sound effect processing solutions based on machine learning to distinguish music types in audio data and perform sound enhancement processing based on the determined music types.

[0004] However, there are still some problems with music classification and sound effect processing solutions based on machine learning: First, the hardware requirements are high, requiring powerful computing chips and high storage requirements. This is because machine learning algorithms usually require strong computing power to process large amounts of data and load model operations, and often rely on high-performance CPUs (Central Processing Units), GPUs (Graphics Processing Units) or dedicated AI (Artificial Intelligence) chips. At the same time, a large amount of training data, model parameters, etc. need to be stored during the training and operation of machine learning models, which places high demands on the capacity and read and write speed of storage devices. Second, in order to achieve complex operations such as large-scale matrix operations and neural network iterations, it is often necessary to equip complex circuit systems, such as multi-core processors, caches, etc., which consume a lot of power when running. Third, the real-time and interpretability are poor. In real-time processing scenarios, it is difficult to respond quickly and accurately to dynamically changing audio signals, and it is easily affected by problems such as data noise and model overfitting. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an audio data processing method and an electronic device, a storage medium, and a computer program product, which are used to classify audio data and enhance sound effects with low hardware requirements and resource consumption.

[0006] On the one hand, the present application provides a method for processing audio data, comprising:

[0007] Extract audio feature information from the target audio data according to the feature extraction strategy corresponding to the specified classification algorithm;

[0008] Classifying the audio feature information according to the specified classification algorithm to obtain the music type of the target audio data;

[0009] A sound enhancement strategy matching the music type is called to perform sound enhancement processing on the target audio data; wherein the sound enhancement strategy refers to an adjustment method of an equalizer, a reverberation mode, and a bass enhancement function.

[0010] In one embodiment, the specified classification algorithm is a classification algorithm based on spectrum energy distribution;

[0011] The step of extracting audio feature information from the target audio data according to the feature extraction strategy corresponding to the specified classification algorithm includes:

[0012] Extracting low-frequency energy, medium-frequency energy and high-frequency energy from the target audio data respectively according to a preset low-frequency range, a preset medium-frequency range and a preset high-frequency range;

[0013] According to the low-frequency energy, the intermediate-frequency energy and the high-frequency energy, a low-frequency energy ratio, an intermediate-frequency energy ratio and a high-frequency energy ratio are calculated respectively as audio feature information.

[0014] In one embodiment, the classifying the audio feature information according to the specified classification algorithm to obtain the music type of the target audio data includes:

[0015] Calculate energy ratio similarity with energy ratio templates corresponding to each music type according to low-frequency energy ratio, mid-frequency energy ratio and high-frequency energy ratio of the target audio data to obtain multiple energy ratio similarities;

[0016] A maximum energy proportion similarity is selected from the multiple energy proportion similarities, and the music type corresponding to the maximum energy proportion similarity is used as the music type of the target audio data.

[0017] In one embodiment, the specified classification algorithm is a classification algorithm based on a combination of spectral features;

[0018] The step of extracting audio feature information from the target audio data according to the feature extraction strategy corresponding to the specified classification algorithm includes:

[0019] Peak frequency, bandwidth, spectrum centroid and zero-crossing rate are extracted from the target audio data as audio feature information.

[0020] In one embodiment, the classifying the audio feature information according to the specified classification algorithm to obtain the music type of the target audio data includes:

[0021] Calculating feature similarities based on the peak frequency, bandwidth, spectrum centroid and zero-crossing rate of the target audio data and the feature information combination corresponding to each music type to obtain multiple feature similarities; wherein the feature information combination includes a specified peak frequency range, a specified bandwidth range, a specified spectrum centroid range and a specified zero-crossing rate range;

[0022] A maximum feature similarity is selected from the multiple feature similarities, and the music type corresponding to the maximum feature similarity is used as the music type of the target audio data.

[0023] In one embodiment, the specified classification algorithm is a classification algorithm based on spectrum template matching;

[0024] The step of extracting audio feature information from the target audio data according to the feature extraction strategy corresponding to the specified classification algorithm includes:

[0025] Spectral feature curves in different frequency bands are extracted from the target audio data as audio feature information.

[0026] In one embodiment, the classifying the audio feature information according to the specified classification algorithm to obtain the music type of the target audio data includes:

[0027] According to the spectrum characteristic curves of the target audio data in different frequency bands, the spectrum curve similarity is calculated with the spectrum template corresponding to each music type to obtain multiple spectrum curve similarities;

[0028] A maximum frequency spectrum curve similarity is selected from the multiple frequency spectrum curve similarities, and the music type corresponding to the maximum frequency spectrum curve similarity is used as the music type of the target audio data.

[0029] In one embodiment, calling a sound effect enhancement strategy that matches the music type to perform sound effect enhancement processing on the target audio data includes:

[0030] If the music type of the target audio data is pop music, the first low-frequency band of the equalizer is enhanced, the first intermediate-frequency band remains unchanged, the first high-frequency band is enhanced, the hall reverberation mode is selected, and the bass enhancement function is turned off;

[0031] If the music type of the target audio data is rock music, the second low frequency band of the equalizer is boosted, the second mid-frequency band is boosted, the second high frequency band is attenuated, the indoor reverberation mode or the stadium reverberation mode is selected, and the bass enhancement function is turned on;

[0032] If the music type of the target audio data is jazz, the third low-frequency band of the equalizer is boosted, the third mid-frequency band is attenuated, the third high-frequency band is boosted, a small room reverberation mode or a plate reverberation mode is selected, and a bass enhancement function is turned on;

[0033] If the music type of the target audio data is classical music, the fourth low frequency band of the equalizer is boosted, the fourth high frequency band is boosted, a large concert hall reverberation mode is selected, and a bass enhancement function is turned off;

[0034] If the music type of the target audio data is electronic music, the fifth low-frequency band of the equalizer is boosted, the fourth mid-frequency band is boosted, the fifth high-frequency band is attenuated, the space reverberation mode or the delay reverberation mode is selected, and the bass enhancement function is turned on.

[0035] On the other hand, the present application provides an electronic device, the electronic device comprising:

[0036] processor;

[0037] a memory for storing processor-executable instructions;

[0038] Wherein, the processor is configured to execute the above-mentioned audio data processing method.

[0039] In addition, the present application provides a computer-readable storage medium, which stores a computer program. The computer program can be executed by a processor to complete the above-mentioned audio data processing method.

[0040] Furthermore, the present application provides a computer program product, including a computer program / instruction, which implements the above-mentioned audio data processing method when executed by a processor.

[0041] The present application scheme can achieve audio data classification and sound enhancement without the help of machine learning, reducing the hardware requirements and resource consumption of audio data processing, and responding quickly and accurately to dynamically changing audio signals in real-time processing scenarios, while avoiding the influence of problems such as model overfitting. The audio classification and sound enhancement process is highly interpretable. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solution of the embodiments of the present application, the drawings required for use in the embodiments of the present application are briefly introduced below.

[0043] Figure 1 A schematic diagram of an application scenario of the method for processing audio data provided in one embodiment of the present application;

[0044] Figure 2 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application;

[0045] Figure 3 A schematic diagram of the structure of an electronic device provided in another embodiment of the present application;

[0046] Figure 4 A flowchart of a method for processing audio data provided by an embodiment of the present application;

[0047] Figure 5 A block diagram of an audio data processing device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0049] Similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0050] Figure 1 Schematic diagram of an application scenario of the method for processing audio data provided in an embodiment of the present application. Figure 1 As shown, the application scenario includes a client 20 and a server 30; the client 20 can be a wireless headset (such as a wireless Bluetooth headset), a wireless speaker, a smart speaker, a home theater, a smart glasses and other audio playback devices; the server 30 can be a mobile phone, a tablet computer, a host, a music player and other devices; the server 30 can output audio data, so that the client 20 receives the audio data and plays the audio.

[0051] like Figure 2 As shown, this embodiment provides an electronic device 1, including: at least one processor 11 and a memory 12, Figure 2 A processor 11 is taken as an example. The processor 11 and the memory 12 are connected via a bus 10, and the memory 12 stores instructions that can be executed by the processor 11. The instructions are executed by the processor 11 so that the electronic device 1 can execute all or part of the process of the method in the following embodiment. In one embodiment, the electronic device 1 can be the above-mentioned client 20 or server 30, which is used to execute the method for processing audio data. In other words, in the aforementioned application scenario, as long as one of the client 20 and the server 30 executes the method for processing audio data, the playback effect of enhanced sound effect can be achieved.

[0052] The memory 12 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable red-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0053] See also Figure 3 , is a structural diagram of an electronic device provided in another embodiment of the present application. When the electronic device is the aforementioned client 20, it may include an audio pickup 310, an embedded real-time operating system 320, and a speaker 330. The audio pickup 310 is used to obtain the original data of the audio signal. The embedded real-time operating system is used to calculate the time-frequency characteristics of the audio signal, classify the audio signal, and perform sound enhancement processing according to the classification results. The speaker 330 is used to play the audio signal after the sound effect is enhanced.

[0054] The present application also provides a computer-readable storage medium, which stores a computer program. The computer program can be executed by the processor 11 to complete the audio data processing method provided by the present application.

[0055] The present application also provides a computer program product, including a computer program / instruction, which implements the audio data processing method provided in the present application when the computer program / instruction is executed by a processor.

[0056] See also Figure 4 , is a flow chart of a method for processing audio data provided by an embodiment of the present application, such as Figure 4 As shown, the method may include the following steps 410 to 430.

[0057] Step 410: extracting audio feature information from the target audio data according to a feature extraction strategy corresponding to a specified classification algorithm.

[0058] In this solution, a designated classification algorithm for distinguishing music types can be pre-configured on an electronic device (such as a wireless Bluetooth headset) that needs to execute the audio data processing method. The designated classification algorithm is provided with a corresponding feature extraction strategy, which indicates the extraction method and extraction content of audio feature information from the target audio data. Here, the designated classification algorithm can be any one of a classification algorithm based on spectral energy distribution, a classification algorithm based on spectral feature combination, a classification algorithm based on spectral template matching, and the like.

[0059] The target audio data is the audio data to be processed. For example, a piece of music currently being played by the electronic device is the target audio data.

[0060] After obtaining the target audio data, feature extraction can be performed on it according to a feature extraction strategy corresponding to a specified classification algorithm, thereby obtaining audio feature information.

[0061] Step 420: Classify the audio feature information according to a specified classification algorithm to obtain the music type of the target audio data.

[0062] After obtaining audio feature information adapted to a specified classification algorithm, the audio feature information may be classified to determine the music type of the target audio data, wherein the music type may include pop music, rock music, jazz, classical music, electronic music, and the like.

[0063] Step 430: Call a sound enhancement strategy that matches the music type to perform sound enhancement processing on the target audio data; wherein the sound enhancement strategy refers to the adjustment method of the equalizer, reverberation mode and bass enhancement function.

[0064] After determining the music type of the target audio data, the corresponding sound enhancement strategy can be called to adjust the equalizer, reverberation mode, bass enhancement function and other indicators to enhance the sound effect of the target audio data. Here, the adjustment of the equalizer may include the boost or attenuation operation of different frequency bands; the adjustment of the reverberation mode includes selecting the reverberation mode, setting the reverberation time, etc.; the adjustment of the bass enhancement function may be turning on or off the bass enhancement function, etc.

[0065] In one embodiment, the processing of the target audio data may be performed periodically, for example, the music type distinction and sound effect enhancement processing may be performed every 5 seconds. In this case, if a song involves multiple music types, different sound effect enhancement methods may be used in different sections of the song to ensure the best overall playback effect.

[0066] Through the above measures, audio data classification and sound enhancement can be achieved without the help of machine learning solutions, which reduces the hardware requirements and resource consumption of audio data processing, and responds quickly and accurately to dynamically changing audio signals in real-time processing scenarios, while avoiding the influence of problems such as model overfitting. The audio classification and sound enhancement process is highly interpretable.

[0067] In one embodiment, the designated classification algorithm is a classification algorithm based on spectral energy. In this case, when executing step 410, low-frequency energy, medium-frequency energy and high-frequency energy can be extracted from the target audio data according to a preset low-frequency range, a preset medium-frequency range and a preset high-frequency range, respectively.

[0068] Among them, the low frequency range, the middle frequency range and the high frequency range can be configured according to the application needs, and can be flexibly adjusted according to the sound processing effect during the application process. For example, the low frequency range is 20Hz (hertz) to 200Hz; the middle frequency range is 200Hz to 2kHz (kilohertz); and the high frequency range is 2kHz to 20kHz.

[0069] The target audio data is divided into the aforementioned low frequency range, medium frequency range and high frequency range by a frequency divider, and then for each frequency band, the energy corresponding to all sampling points in the frequency band is calculated to obtain low frequency energy, medium frequency energy and high frequency energy.

[0070] Exemplarily, the energy of any frequency band is calculated by the following formula (1):

[0071]

[0072] Among them, E k represents the energy of the kth frequency band; a i Represents the amplitude of the i-th sampling point.

[0073] By calculating the square of the amplitude of each sampling point in any frequency band and calculating the sum of the squares of all sampling points, the energy of the frequency band can be obtained.

[0074] After obtaining the low-frequency energy, the middle-frequency energy and the high-frequency energy, the low-frequency energy ratio, the middle-frequency energy ratio and the high-frequency energy ratio are calculated respectively as audio feature information.

[0075] Exemplarily, the energy proportion of any frequency band can be calculated by the following formula (2):

[0076]

[0077] Among them, P k Indicates the energy proportion of the kth frequency band; E x Represents the energy of the kth frequency band.

[0078] By calculating the total energy of low-frequency energy, medium-frequency energy and high-frequency energy, the low-frequency energy ratio can be obtained by dividing the low-frequency energy by the total energy, the medium-frequency energy ratio can be obtained by dividing the medium-frequency energy by the total energy, and the high-frequency energy ratio can be obtained by dividing the high-frequency energy by the total energy.

[0079] Through the above measures, the low-frequency energy proportion, mid-frequency energy proportion and high-frequency energy proportion can be calculated for the target audio data as audio feature information for subsequent classification by a classification algorithm based on spectral energy distribution.

[0080] In one embodiment, if the specified classification algorithm can be a classification algorithm based on spectral energy distribution, when executing step 420 to classify the audio feature information, the energy proportion similarity can be calculated based on the low-frequency energy proportion, mid-frequency energy proportion and high-frequency energy proportion of the target audio data and the energy proportion template corresponding to each music type.

[0081] Among them, the energy proportion template can record the low-frequency energy proportion range, the medium-frequency energy proportion range and the high-frequency energy proportion range of the music type.

[0082] For example, if the music type is jazz, the low-frequency energy proportion in the energy proportion template ranges from 30% to 50%, the middle-frequency energy proportion ranges from 30% to 40%, and the high-frequency energy proportion ranges from 20% to 30%.

[0083] When calculating the energy proportion similarity, methods such as Euclidean distance and cosine similarity can be used for calculation.

[0084] For example, if the low frequency energy proportion of the target audio data is 45%, the mid-frequency energy proportion is 30%, and the high frequency energy proportion is 25%, then the energy proportion similarity between it and the energy proportion template of jazz music can be calculated. First, the difference between the energy proportion of each frequency band and the energy proportion range is calculated respectively, and then the similarity is determined by multiple differences.

[0085] The difference between the low-frequency energy proportion and the low-frequency energy proportion range (denoted as diff1) is:

[0086]

[0087] The difference between the intermediate frequency energy proportion and the intermediate frequency energy proportion range (denoted as diff2) is:

[0088]

[0089] The difference between the high-frequency energy proportion and the high-frequency energy proportion range (denoted as diff3) is:

[0090]

[0091] Exemplarily, the energy proportion similarity is determined by the following formula (3) using the above differences:

[0092]

[0093] Among them, similarity is the energy ratio similarity; diff i is the i-th difference; weight i is the weight corresponding to the i-th difference.

[0094] After calculating the energy share similarities between the target audio data and pop music, rock music, jazz, classical music, and electronic music, the maximum energy share similarity is selected from these energy share similarities. At this time, the spectral energy distribution of the target audio data is most similar to the spectral energy distribution of the music type corresponding to the maximum energy share similarity. Therefore, the music type corresponding to the maximum energy share similarity can be used as the music type of the target audio data.

[0095] In one embodiment, the weights of different frequency bands can be dynamically adjusted during song playback. For example, if the user indicates on the user interface of the electronic device that the current audio effect is too noisy or too dull, the weights of each frequency band can be dynamically adjusted based on the user's feedback. If the user feedback is that the sound is too noisy, it means that the high-frequency energy may be too high, and the weight of the high-frequency energy can be increased.

[0096] By dynamically adjusting the weights, the accuracy of music type classification can be improved.

[0097] In one embodiment, the designated classification algorithm is a classification algorithm based on a combination of spectral features. In this case, when executing step 410, peak frequency, bandwidth, spectral centroid and zero-crossing rate can be extracted from the target audio data as audio feature information.

[0098] For the target audio data, the i-th frame signal x is transformed by framing and Fourier transform. i (n) Obtain spectrum information X from time domain transformation i (k), and use the spectrum information to calculate the peak frequency, bandwidth, spectrum centroid and zero-crossing rate. Where n ranges from 0, 1, 2, ..., N-1. k ranges from 0, 1, 2, ..., N-1.

[0099] The peak frequency indicates the frequency corresponding to the frequency component with the largest amplitude in the spectrum. Find the index k corresponding to the maximum amplitude of the spectrum from the spectrum information. max , and then according to the relationship between discrete frequency and actual frequency, the peak frequency is obtained f s is the sampling frequency.

[0100] Bandwidth is the difference between the two frequencies where the peak amplitude drops to -3dB (decibels).

[0101] Spectral Centroid is the frequency of the "center of mass" of the spectrum.

[0102]

[0103] Zero Crossing Rate refers to the number of times a signal crosses the zero point from positive to negative or from negative to positive per unit time. It is usually used to measure the frequency of change or activity of a signal.

[0104] In one embodiment, if the designated classification algorithm is a classification algorithm based on a spectral feature combination, when executing step 420 to classify the audio feature information, the feature similarity can be calculated with the feature information combination corresponding to each music type according to the peak frequency, bandwidth, spectral centroid and zero crossing rate of the target audio data, wherein the feature information combination includes a designated peak frequency range, a designated bandwidth range, a designated spectral centroid range and a designated zero crossing rate range.

[0105] Exemplarily, in the characteristic information combination of pop music, the peak frequency range is specified to be 1000 Hz to 3000 Hz, the bandwidth range is specified to be 1500 Hz to 3000 Hz, the spectrum centroid range is specified to be 1500 Hz to 2500 Hz, and the zero-crossing rate range is specified to be 20 times / second to 50 times / second.

[0106] In the characteristic information combination of rock music, the peak frequency is specified to range from 200 Hz to 500 Hz, the bandwidth is specified to range from 2000 Hz to 5000 Hz, the spectrum centroid is specified to range from 1000 Hz to 2000 Hz, and the zero crossing rate is specified to range from 80 times / second to 150 times / second.

[0107] In the characteristic information combination of jazz music, the peak frequency is specified to range from 800 Hz to 2500 Hz, the bandwidth is specified to range from 1200 Hz to 2500 Hz, the spectrum centroid is specified to range from 1200 Hz to 2000 Hz, and the zero crossing rate is specified to range from 40 times / second to 80 times / second.

[0108] In the characteristic information combination of classical music, the peak frequency range is specified to be 20 Hz to 10 kHz, the bandwidth range is specified to be 100 Hz to 10 kHz, the spectrum centroid range is specified to be 500 Hz to 3 kHz, and the zero crossing rate range is specified to be 100 times / second to 1000 times / second.

[0109] In the characteristic information combination of electronic music, the peak frequency range is specified to be 200 Hz to 5000 Hz, the bandwidth range is specified to be above 3000 Hz, the spectrum centroid range is specified to be 1500 Hz to 3000 Hz, and the zero crossing rate range is specified to be 60 times / second to 120 times / second.

[0110] The feature similarity is calculated by methods such as Euclidean distance or cosine similarity. In the application process, the weights of different audio feature information in the feature information combination can be dynamically adjusted according to the changes in the spectral features of different sections in the song. For example, the weight of each audio feature information is generally between 0.1 and 0.3. If the change of the spectral centroid in a certain piece of music is found to be more important for distinguishing the type of music, the weight can be adjusted to 0.3 or 0.4 to more accurately calculate the feature similarity for classification.

[0111] After calculating the feature similarities between the target audio data and pop music, rock music, jazz, classical music, and electronic music, the maximum feature similarity is selected from these feature similarities. At this time, the spectral feature combination of the target audio data is most similar to the spectral feature combination of the music type corresponding to the maximum feature similarity. Therefore, the music type corresponding to the maximum feature similarity can be used as the music type of the target audio data.

[0112] In one embodiment, the classification algorithm is designated as a classification algorithm based on spectrum template matching. In this case, when executing step 410, spectrum feature curves in different frequency bands can be extracted from the target audio data as audio feature information. Here, the spectrum feature curve can be an average spectrum energy curve, an average spectrum mean curve, an average spectrum peak curve, etc.

[0113] In one embodiment, if the designated classification algorithm is a classification algorithm based on spectrum template matching, when executing step 420 to classify the audio feature information, the spectrum curve similarity can be calculated based on the spectrum feature curve of the target audio data in different frequency bands and the spectrum template corresponding to each music type to obtain multiple spectrum curve similarities.

[0114] Here, the spectrum template corresponding to each music type contains spectrum feature curves of different frequency bands. The spectrum feature curve can be an average spectrum energy curve, an average spectrum mean curve, an average spectrum peak curve, etc. The type of spectrum feature curve in the spectrum template determines the type of spectrum feature curve generated for the target audio data in the feature extraction stage.

[0115] For any music genre, a large number of representative works of different singers, bands, and periods under this music genre can be collected as music samples to ensure the diversity and comprehensiveness of the data. By aggregating the features of a large number of music samples under the same music genre and then performing statistical analysis, the spectrum characteristic curve of this music genre in each frequency band can be obtained. In addition, abnormal data can be identified and processed to ensure that a valid spectrum characteristic curve is obtained. For example, unreasonable feature values ​​caused by audio recording problems or extreme music styles can be processed by data smoothing, deleting abnormal points, etc. Then, the spectrum characteristic curve is generated with the processed feature values.

[0116] After generating the spectral feature curve of the spectral template for any type of music, the accuracy and generalization ability of the spectral template can be verified through a part of test music samples that are not involved in the establishment of the spectral template. According to the verification results, the spectral target can be adjusted and optimized, such as adjusting the frequency band division method, feature calculation method or aggregation statistics.

[0117] After calculating the spectral curve similarities of the target audio data with pop music, rock music, jazz, classical music, and electronic music, the maximum spectral curve similarity is selected from these spectral curve similarities. At this time, the spectral curve trend of the target audio data is most similar to the spectral curve trend of the music type corresponding to the maximum spectral curve similarity. Therefore, the music type corresponding to the maximum spectral curve similarity can be used as the music type of the target audio data.

[0118] In one embodiment, during the process of executing step 430 to perform sound enhancement processing on the target audio data, if the music type of the target audio data is pop music, the first low frequency band of the equalizer can be enhanced, the first intermediate frequency band remains unchanged, and the first high frequency band is enhanced. Here, the first low frequency band, the first intermediate frequency band, and the first high frequency band can be configured as needed, and the adjustment range is an empirical value. Exemplarily, the first low frequency band is 20Hz to 200Hz, which can be appropriately increased by 2dB to 3dB to enhance the sense of rhythm and heaviness; the first intermediate frequency band is 200Hz to 2kHz, which can be kept unchanged to ensure that the sound of life and musical instruments is clear and natural; the first high frequency band is 2kHz to 20kHz, which can be increased by 2dB to 3dB to make the sound brighter and more transparent, and increase the vitality of the song. Select the hall reverberation mode, and set the reverberation time to about 1.5 seconds to 2 seconds, so that the sound creates an open and grand sense of space, making the song more atmospheric. In addition, the bass enhancement function can be turned off to avoid covering up the sound of other frequency bands.

[0119] In one embodiment, during the process of executing step 430 to perform sound enhancement processing on the target audio data, if the music type of the target audio data is rock music, the second low frequency band of the equalizer can be boosted, the second intermediate frequency band can be boosted, and the second high frequency band can be attenuated. Here, the second low frequency band, the second intermediate frequency band, and the second high frequency band can be configured as needed, and the adjustment range is an empirical value. Exemplarily, the second low frequency band is 60Hz to 250Hz, which can be greatly boosted to increase the low-frequency impact of electric guitar, bass and drums; the second intermediate frequency band is 3kHz to 5kHz, which can be appropriately boosted to make the distorted tone of the guitar sharper and more aggressive; the second high frequency band is a band above 8kHz, which can be attenuated to reduce unnecessary noise and harshness. Select the indoor reverberation mode or the stadium reverberation mode, and set the reverberation time to about 1.8 seconds to 2.5 seconds to create a sense of vast space in a large performance venue and enhance the shock of the music. In addition, you can turn on the bass increase function to greatly enhance the bass, making the sound of the bass drum and bass more explosive and enhancing the impact of the live listening experience.

[0120] In one embodiment, during the process of executing step 430 to perform sound enhancement processing on the target audio data, if the music type of the target audio data is jazz, the third low frequency band of the equalizer can be enhanced, the third intermediate frequency band can be attenuated, and the third high frequency band can be enhanced. Here, the third low frequency band, the third intermediate frequency band, and the third high frequency band can be configured as needed, and the adjustment range is an empirical value. Exemplarily, the third low frequency band is 150Hz to 300Hz, which can be appropriately enhanced to enhance the resonance and warmth of the instrument; the third intermediate frequency band is 4kHz to 6kHz, which can be attenuated to reduce the harshness and make the sound softer; the third high frequency band is 8kHz to 10kHz, which can be slightly enhanced to enhance some airiness. Selecting the small room reverberation mode or the plate reverberation mode, and setting the reverberation time to about 1 second to 1.2 seconds can simulate the sense of space of a small club or recording studio where jazz is played, making the sound more live and intimate. In addition, the bass enhancement function can be turned on to moderately enhance the bass.

[0121] In one embodiment, during the process of executing step 430 to perform sound enhancement processing on the target audio data, if the music type of the target audio data is classical music, the fourth low-frequency band of the equalizer can be enhanced and the fourth high-frequency band can be enhanced. Here, the fourth low-frequency band and the fourth high-frequency band are configured as needed, and the adjustment amplitude is an empirical value. Exemplarily, the relative balance of the entire frequency band is maintained, the fourth low-frequency band is 200Hz to 500Hz, which can be slightly enhanced to increase the warmth and thickness of string instruments and wind instruments; the fourth high-frequency band is a 10kHz peripheral band, which can be slightly enhanced to highlight the high-frequency details and glossiness of the instruments. Select the large concert hall reverberation mode, and set the reverberation time to about 2 seconds to 3 seconds to simulate the spatial effect of a real concert hall, so that the audience can feel the grand scene and spatial atmosphere of classical music performance. In addition, the bass enhancement function can be turned off to avoid affecting the overall balance.

[0122] In one embodiment, during the process of executing step 430 to perform sound enhancement processing on the target audio data, if the music type of the target audio data is electronic music, the fifth low frequency band of the equalizer can be boosted, the fourth intermediate frequency band can be boosted, and the fifth high frequency band can be attenuated. Here, the fifth low frequency band, the fourth intermediate frequency band, and the fifth high frequency band can be configured as needed, and the adjustment range is an empirical value. Exemplarily, the fifth low frequency band is 20Hz to 80Hz, which can be greatly boosted to enhance the shock of the heavy bass; the fourth intermediate frequency band is 2kHz to 4kHz, which can be boosted to highlight the main timbre of the synthesizer and the high-frequency details of some percussion instruments; the fifth high frequency band is 6kHz to 8kHz, which can be appropriately attenuated to avoid the sound being too sharp. Select the space reverberation mode or the delay reverberation mode, and the reverberation time can be determined according to the specific style and rhythm, for example: the reverberation time of fast-paced electronic music is about 0.8 seconds to 1.2 seconds, and the reverberation time of slow-paced electronic music is about 1.5 seconds to 2 seconds. In addition, you can turn on the bass enhancement function to significantly increase the bass, making the heavy bass the dominant force in the music and enhancing the atmosphere and rhythm of the dance floor.

[0123] In summary, this solution can identify the music type based on the time domain characteristics of the input audio data and provide different sound enhancement solutions for different music types, which can achieve a more ideal playback effect. This solution does not require large-scale matrix operation, neural network iteration and other complex operations. It is mainly based on basic mathematical operations and signal processing algorithms, which significantly reduces the requirements for hardware computing energy and reduces power consumption.

[0124] Figure 5 is a block diagram of an audio data processing device according to an embodiment of the present invention. Figure 5 As shown, the device may include:

[0125] An extraction module 510, configured to extract audio feature information from target audio data according to a feature extraction strategy corresponding to a specified classification algorithm;

[0126] A classification module 520, configured to classify the audio feature information according to the specified classification algorithm to obtain the music type of the target audio data;

[0127] The processing module 530 is used to call the sound enhancement strategy that matches the music type to perform sound enhancement processing on the target audio data; wherein the sound enhancement strategy refers to the adjustment method of the equalizer, reverberation mode and bass enhancement function.

[0128] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned audio data processing method, which will not be repeated here.

[0129] In several embodiments provided in the present application, the disclosed devices and methods may also be implemented in other ways. The device embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to the multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment or a part of a code, and a module, a program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or the flowchart, and the combination of boxes in the block diagram and / or the flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0130] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0131] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

Claims

1. A method for processing audio data, characterized in that: include: Extract audio feature information from the target audio data according to the feature extraction strategy corresponding to the specified classification algorithm; Classifying the audio feature information according to the specified classification algorithm to obtain the music type of the target audio data; A sound enhancement strategy matching the music type is called to perform sound enhancement processing on the target audio data; wherein the sound enhancement strategy refers to an adjustment method of an equalizer, a reverberation mode, and a bass enhancement function.

2. The method according to claim 1, characterized in that The specified classification algorithm is a classification algorithm based on spectrum energy distribution; The step of extracting audio feature information from the target audio data according to the feature extraction strategy corresponding to the specified classification algorithm includes: According to a preset low frequency range, a preset intermediate frequency range and a preset high frequency range, respectively extracting low frequency energy, intermediate frequency energy and high frequency energy from the target audio data; According to the low-frequency energy, the intermediate-frequency energy and the high-frequency energy, a low-frequency energy proportion, an intermediate-frequency energy proportion and a high-frequency energy proportion are calculated respectively as audio feature information.

3. The method according to claim 2, characterized in that The classifying the audio feature information according to the specified classification algorithm to obtain the music type of the target audio data includes: Calculate energy ratio similarity with energy ratio templates corresponding to each music type according to low-frequency energy ratio, mid-frequency energy ratio and high-frequency energy ratio of the target audio data to obtain multiple energy ratio similarities; A maximum energy proportion similarity is selected from the multiple energy proportion similarities, and the music type corresponding to the maximum energy proportion similarity is used as the music type of the target audio data.

4. The method according to claim 1, characterized in that: The specified classification algorithm is a classification algorithm based on a combination of spectral features; The step of extracting audio feature information from the target audio data according to the feature extraction strategy corresponding to the specified classification algorithm includes: Peak frequency, bandwidth, spectrum centroid and zero-crossing rate are extracted from the target audio data as audio feature information.

5. The method according to claim 4, characterized in that The classifying the audio feature information according to the specified classification algorithm to obtain the music type of the target audio data includes: Calculating feature similarities based on the peak frequency, bandwidth, spectrum centroid and zero-crossing rate of the target audio data and the feature information combination corresponding to each music type to obtain multiple feature similarities; wherein the feature information combination includes a specified peak frequency range, a specified bandwidth range, a specified spectrum centroid range and a specified zero-crossing rate range; A maximum feature similarity is selected from the multiple feature similarities, and the music type corresponding to the maximum feature similarity is used as the music type of the target audio data.

6. The method according to claim 1, characterized in that The specified classification algorithm is a classification algorithm based on spectrum template matching; The step of extracting audio feature information from the target audio data according to the feature extraction strategy corresponding to the specified classification algorithm includes: Spectral feature curves in different frequency bands are extracted from the target audio data as audio feature information.

7. The method according to claim 6, characterized in that The classifying the audio feature information according to the specified classification algorithm to obtain the music type of the target audio data includes: According to the spectrum characteristic curves of the target audio data in different frequency bands, the spectrum curve similarity is calculated with the spectrum template corresponding to each music type to obtain multiple spectrum curve similarities; A maximum frequency spectrum curve similarity is selected from the multiple frequency spectrum curve similarities, and the music type corresponding to the maximum frequency spectrum curve similarity is used as the music type of the target audio data.

8. The method according to claim 1, characterized in that The calling of the sound effect enhancement strategy matching the music type to perform sound effect enhancement processing on the target audio data includes: If the music type of the target audio data is pop music, the first low-frequency band of the equalizer is enhanced, the first intermediate-frequency band remains unchanged, the first high-frequency band is enhanced, the hall reverberation mode is selected, and the bass enhancement function is turned off; If the music type of the target audio data is rock music, the second low frequency band of the equalizer is boosted, the second mid-frequency band is boosted, the second high frequency band is attenuated, the indoor reverberation mode or the stadium reverberation mode is selected, and the bass enhancement function is turned on; If the music type of the target audio data is jazz, the third low-frequency band of the equalizer is boosted, the third mid-frequency band is attenuated, the third high-frequency band is boosted, a small room reverberation mode or a plate reverberation mode is selected, and a bass enhancement function is turned on; If the music type of the target audio data is classical music, the fourth low frequency band of the equalizer is boosted, the fourth high frequency band is boosted, a large concert hall reverberation mode is selected, and a bass enhancement function is turned off; If the music type of the target audio data is electronic music, the fifth low-frequency band of the equalizer is boosted, the fourth mid-frequency band is boosted, the fifth high-frequency band is attenuated, the space reverberation mode or the delay reverberation mode is selected, and the bass enhancement function is turned on.

9. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing processor-executable instructions; The processor is configured to execute the audio data processing method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program can be executed by a processor to complete the audio data processing method according to any one of claims 1 to 8.

11. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the method for processing audio data described in any one of claims 1 to 8 is implemented.

Images