Audio signal feature extraction method and device, storage medium and electronic equipment

Abstract

The invention provides an audio signal feature extraction method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining an audio signal; extracting an initial Mel spectrum of the audio signal; a weight matrix of the initial Mel spectrum is calculated, and the weight matrix is used for compressing the number of time frames in the initial Mel spectrum; and performing matrix multiplication on the weight matrix and the initial Mel spectrum to obtain a simplified Mel spectrum. According to the method and the device, redundant data in the audio signal is reduced, so that the data volume of audio features is reduced, the accuracy and generalization ability of the model are improved, the calculation consumption of the model is reduced, and the efficiency and the performance of an audio analysis task are improved.

Description

Technical Field

[0001] This invention relates to the field of audio processing, and more specifically, to a method and apparatus for feature extraction of audio signals, a storage medium, and an electronic device. Background Technology

[0002] In related technologies, audio processing tasks in deep learning involve extremely complex features in raw audio signals, making it difficult to accurately detect temporal information through simple waveform comparison alone. Traditional audio feature extraction methods typically divide long audio signals into short frames of several milliseconds, and then extract spectral features corresponding to different frequencies in each frame using short-time Fourier transform. This method exhibits strong feature capture capabilities in the frequency domain, but most audio frames have little impact on the task processing results. The calculation result is determined by only a small portion of the input data, and these frames, which do not contain important information, severely hinder the processing efficiency of audio tasks.

[0003] There are currently no effective solutions to the aforementioned problems in the relevant technologies. Summary of the Invention

[0004] This invention provides a method and apparatus for feature extraction of audio signals, a storage medium, and an electronic device.

[0005] According to an embodiment of the present invention, a feature extraction method for an audio signal is provided, comprising: acquiring an audio signal; extracting an initial Mel spectrum of the audio signal; calculating a weight matrix of the initial Mel spectrum, wherein the weight matrix is ​​used to compress the number of time frames in the initial Mel spectrum; and performing matrix multiplication on the weight matrix and the initial Mel spectrum to obtain the simplified Mel spectrum.

[0006] Optionally, extracting the initial Mel spectrum of the audio signal includes: performing a short-time Fourier transform on the audio signal to obtain time-frequency features; filtering the time-frequency features using a Mel filter bank to obtain spectral energy; and taking the logarithm of the spectral energy to obtain the initial Mel spectrum.

[0007] Optionally, calculating the weight matrix of the initial Mel spectrum includes: calculating the score value of each input time frame in the initial Mel spectrum, wherein the score value is used to characterize the importance of the corresponding input time frame, the initial Mel spectrum includes multiple input time frames, and the score value takes a value between 0 and 1; generating a weight matrix based on the score values ​​of all input time frames.

[0008] Optionally, calculating the score value for each input time frame in the initial Mel spectrum includes: performing a one-dimensional convolution on each input time frame in the initial Mel spectrum to obtain the feature value of the time frame in the time dimension; generating a 1×T first score vector based on the feature values ​​of all time frames, where T is the number of input time frames in the initial Mel spectrum; obtaining the number t of output time frames of the simplified Mel spectrum; and normalizing the first score vector according to the number t to obtain a second score vector of the initial Mel spectrum, wherein the second score vector includes the score value of each input time frame in the initial Mel spectrum.

[0009] Optionally, normalizing the first score vector according to the quantity t to obtain the second score vector of the initial Mel spectrum includes: normalizing the first score vector using the following formula to obtain the second score vector s of the initial Mel spectrum: Among them, s ′ Let s be the first score vector. ′ k The value of the input time frame k in the first score vector.

[0010] Optionally, generating the weight matrix based on the scores of all input time frames includes: constructing an empty matrix of size t×T, where T is the number of input time frames in the initial Mel spectrum and t is the number of output time frames in the simplified Mel spectrum; calculating the cumulative sum of the scores of all input time frames; and calculating W for each element in the empty matrix based on the cumulative sum. i,j The element values ​​are used to obtain the weight matrix, where the element W... i,j This represents the contribution of the j-th input time frame of the initial Mel spectrum to the i-th output time frame of the simplified Mel spectrum.

[0011] Optionally, each element W in the empty matrix is ​​calculated based on the cumulative sum. i,j The element values ​​include: for each element W i,j Calculate the score of the initial Mel spectrum from the first input time frame to the j-th input time frame. judge Whether it is true; if Establishment, determination of element W i,j The element value is s j ;like This is not true; determine element W. i,j The element value is 0, s j The score value for the j-th input time frame of the initial Mel spectrum.

[0012] According to another embodiment of the present invention, an audio signal feature extraction apparatus is provided, comprising: an acquisition module for acquiring an audio signal; an extraction module for extracting an initial Mel spectrum of the audio signal; a calculation module for calculating a weight matrix of the initial Mel spectrum, wherein the weight matrix is ​​used to compress the number of time frames in the initial Mel spectrum; and matrix multiplication of the weight matrix and the initial Mel spectrum to obtain the simplified Mel spectrum.

[0013] According to yet another embodiment of the present invention, a storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to execute the steps in any of the above method embodiments when running.

[0014] According to yet another embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0015] This invention acquires an audio signal; extracts the initial Mel spectrum of the audio signal; calculates a weight matrix for the initial Mel spectrum, wherein the weight matrix is ​​used to compress the number of time frames in the initial Mel spectrum; and performs matrix multiplication on the weight matrix and the initial Mel spectrum to obtain the simplified Mel spectrum. By reducing the number of time frames in the initial Mel spectrum through the weight matrix, useless information in the initial Mel spectrum can be compressed. This provides a method for filtering and retaining audio frames with obvious characteristics, reducing redundant data in the audio signal, thereby reducing the amount of audio feature data, improving the accuracy and generalization ability of the model, reducing the computational cost of the model, and improving the efficiency and performance of audio analysis tasks. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0017] Figure 1 This is a flowchart of an audio signal feature extraction method according to an embodiment of the present invention;

[0018] Figure 2 This is a schematic diagram of the initial Mel spectrum of the audio signal extracted in an embodiment of the present invention;

[0019] Figure 3 This is a schematic diagram of the Mel spectrum feature simplification process in an embodiment of the invention;

[0020] Figure 4This is a structural block diagram of an audio signal feature extraction device according to an embodiment of the present invention;

[0021] Figure 5 This is a structural diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all of them. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present application can be combined with each other.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] Example 1

[0025] This embodiment provides a method for feature extraction of audio signals, which can be executed in a voice recorder, smart speaker, mobile phone, server, or similar audio processing device. Figure 1 This is a flowchart of an audio signal feature extraction method according to an embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0026] Step S101: Acquire audio signal;

[0027] The audio signal in this embodiment can be a sound signal collected by an audio device, or it can be other audio signals that need to be feature extracted and processed.

[0028] Step S102: Extract the initial Mel spectrum of the audio signal;

[0029] The feature extraction model in this embodiment takes the initial Mel spectrum as input and the compressed simplified Mel spectrum as output. The initial Mel spectrum includes multiple input time frames, and the simplified Mel spectrum includes multiple output time frames. The number of input time frames is T, and the number of output time frames is t, where T is greater than t.

[0030] Optionally, t can be determined based on the preset compression ratio and the preset number of time frames after compression.

[0031] Step S103: Calculate the weight matrix of the initial Mel spectrum, wherein the weight matrix is ​​used to compress the number of time frames in the initial Mel spectrum;

[0032] Optionally, the weight matrix has a size of t×T. Taking the weight matrix as matrix W as an example, the shape of matrix W is t*T. Through matrix W, the shape of the initial Mel spectrum can be compressed from F*T to F*t, thereby reducing the number of time frames.

[0033] Step S104: Perform matrix multiplication on the weight matrix and the initial Mel spectrum to obtain the simplified Mel spectrum.

[0034] The weight matrix and the initial Mel spectrum are frame-merged. The generated weight matrix W is then used to perform matrix multiplication on the initial Mel spectrum to complete the time frame merging. The output simplified Mel spectrum O has a shape of F×t, which retains important time frame information, reduces redundant frames, and thus improves classification efficiency.

[0035] The simplified Mel spectrum of this embodiment can be applied to various audio processing scenarios. The output Mel spectrum O is used for subsequent classification tasks. This process ensures that the most important information in the audio is preserved while reducing unnecessary computational burden.

[0036] The solution in this embodiment can be applied to the metaverse (such as virtual digital humans, AR (Augmented Reality) / VR (Virtual Reality) / MR (Mixed Reality) devices, virtual games, virtual animation, live streaming, film and television dramas, and other fields. In an example of an application scenario oriented towards the metaverse, a user creates their own virtual digital human product. For the audio signal output by the user, the solution in this embodiment can be used to extract the features of the audio signal, so that the virtual digital human product can play more accurate speech. In an example of an application scenario of virtual animation, the audio signal of a real actor is collected to dub a virtual character. The audio signal output by the real actor can be collected, and the solution in this embodiment can be used to extract the features of the audio signal, so that the virtual character in the virtual animation can play more accurate speech, and it can also reduce the amount of audio feature data and reduce the loading time of audio resources.

[0037] Through the above steps, an audio signal is acquired; the initial Mel spectrum of the audio signal is extracted; the weight matrix of the initial Mel spectrum is calculated, wherein the weight matrix is ​​used to compress the number of time frames in the initial Mel spectrum; the weight matrix and the initial Mel spectrum are multiplied by matrix to obtain the simplified Mel spectrum. By reducing the number of time frames in the initial Mel spectrum through the weight matrix, useless information in the initial Mel spectrum can be compressed, realizing a method for filtering and retaining audio frames with obvious features, reducing redundant data in the audio signal, thereby reducing the amount of audio feature data, improving the accuracy and generalization ability of the model, reducing the computational cost of the model, and improving the efficiency and performance of the audio analysis task.

[0038] In one embodiment of this example, extracting the initial Mel spectrum of the audio signal includes: performing a short-time Fourier transform on the audio signal to obtain time-frequency features; filtering the time-frequency features using a Mel filter bank to obtain spectral energy; and taking the logarithm of the spectral energy to obtain the initial Mel spectrum.

[0039] In this embodiment, the one-dimensional audio waveform of the input audio signal is first subjected to a short-time fourier transform (STFT) to obtain the time-frequency characteristics of the time-frequency representation. Then, the spectrum is filtered by a Mel filter bank to obtain the spectral energy. Finally, the logarithm of the spectral energy is taken to obtain the initial Mel spectrum.

[0040] Optionally, the initial Mel spectrum is a Mel spectrum graph X with a shape of F×T, where F is the frequency dimension and T is the number of time frames.

[0041] In another embodiment of this example, the audio signal can be pre-emphasized, framed, and windowed before the short-time Fourier transform. The initial Mel spectrum extraction process includes: pre-emphasis, framing, windowing, Fourier transform, Mel filter filtering, and logarithmic transform. Figure 2 This is a schematic diagram of extracting the initial Mel spectrum of an audio signal in an embodiment of the present invention, including:

[0042] Pre-emphasis: Used to enhance the high-frequency components of an audio signal. A first-order filter can be used to pre-emphasize the audio signal. The formula for a first-order filter is as follows:

[0043] y[n] = x[n] - αx[n-1];

[0044] Where x[n] is the original audio signal, y[n] is the pre-emphasized signal, and α is the pre-emphasis coefficient, which can be between 0.95 and 0.99.

[0045] Framing: Dividing the audio signal into multiple frames of length L, with a frame shift of H, to ensure the short-term stationarity of the signal.

[0046] Windowing: A window function is applied to each frame after framing to smooth the features between speech frames. The window function is a Hamming window, and its calculation formula is as follows:

[0047]

[0048] Where N is the length of the frame, and w[n] is the window function value.

[0049] Fourier Transform (FFT): Applying the Fast Fourier Transform (FFT) to the windowed frame converts the time-domain signal into a frequency-domain signal. The calculation formula is as follows:

[0050]

[0051] Where k represents the frequency component, and X(k) is the spectrum value of the transformed frequency domain signal.

[0052] Mel filter filtering: The spectrum is mapped onto the Mel frequency scale through a Mel filter bank. The relationship between the linear frequency f and the Mel frequency m(f) is as follows:

[0053]

[0054] Using these relationships, M triangular Mel filters Hm[k] can be designed. The output of each filter is:

[0055]

[0056] The spectral energy of a frequency domain signal in multiple frequency bands is calculated using a set of Mel filters.

[0057] Logarithmic transformation: The logarithmic value of the spectral energy obtained through the Mel filter is taken to obtain the initial Mel spectrum, so as to simulate the human ear's perception of sound.

[0058] In this embodiment, calculating the weight matrix of the initial Mel spectrum includes:

[0059] S11, calculate the score value of each input time frame in the initial Mel spectrum, wherein the score value is used to characterize the importance of the corresponding input time frame, the initial Mel spectrum includes multiple input time frames, and the score value is between 0 and 1;

[0060] A convolutional neural network based on a residual structure can be used to compute the score for each time frame. The convolutional neural network model consists of a five-layer one-dimensional convolutional network (ResConv1 D), with each convolutional layer followed by batch normalization and a Leaky ReLU activation function. Residual connections are used to improve training stability.

[0061] In one example, calculating the score value for each input time frame in the initial Mel spectrum includes: performing a one-dimensional convolution on each input time frame in the initial Mel spectrum to obtain the feature value of the time frame in the time dimension; generating a 1×T first score vector based on the feature values ​​of all time frames, where T is the number of input time frames in the initial Mel spectrum; obtaining the number t of output time frames of the simplified Mel spectrum; and normalizing the first score vector according to the number t to obtain a second score vector of the initial Mel spectrum, wherein the second score vector includes the score value of each input time frame in the initial Mel spectrum.

[0062] In this example, a convolution operation is first performed on each input time frame in the initial Mel spectrum. Taking the initial Mel spectrum as Mel spectrum X and the input time frame of Mel spectrum X as time frame Xi as an example, after inputting Mel spectrum X, the model performs a one-dimensional convolution on each time frame Xi to extract the feature values ​​in the time dimension. The convolution operation ensures that the output still maintains the same time dimension T as the input.

[0063] After extracting the feature values ​​of each input time frame through convolution operations, an importance score vector s′ of shape 1×T is generated, which is the first score vector, where si′ represents the score of the i-th frame in s′. The score of si′ is restricted to the range [0,1] by the sigmoid function.

[0064] Finally, to ensure that the score matches the output time frame t, the score vector s′ is normalized to obtain the final score vector s. The sum of the normalized scores meets the requirements of the output time dimension t.

[0065] Optionally, normalizing the first score vector according to the quantity t to obtain the second score vector of the initial Mel spectrum includes: normalizing the first score vector using the following formula to obtain the second score vector s of the initial Mel spectrum: Among them, s ′ Let s be the first score vector. ′ k The value of the input time frame k in the first score vector.

[0066] S12, generate a weight matrix based on the scores of all input time frames.

[0067] In one example, generating a weight matrix based on the scores of all input time frames includes: constructing an empty matrix of size t×T, where T is the number of input time frames in the initial Mel spectrum and t is the number of output time frames in the simplified Mel spectrum; calculating the cumulative sum of the scores of all input time frames; and calculating W for each element of the empty matrix based on the cumulative sum. i,j The element values ​​are used to obtain the weight matrix, where the element W... i,j This represents the contribution of the j-th input time frame of the initial Mel spectrum to the i-th output time frame of the simplified Mel spectrum.

[0068] In this example, an empty matrix is ​​first constructed. Based on the normalized frame importance score s, the elements of the empty matrix are assigned values ​​to generate a weight matrix W of shape t×T. Each element W of the matrix... i,j This represents the contribution of the j-th frame of the input Mel spectrum to the i-th frame of the output, and is used for subsequent matrix multiplication operations.

[0069] Optionally, each element W in the empty matrix is ​​calculated based on the cumulative sum. i,j The element values ​​include: for each element W i,j Calculate the score of the initial Mel spectrum from the first input time frame to the j-th input time frame. judge Whether it is true; if Establishment, determination of element W i,j The element value is s j ;like This is not true; determine element W. i,j The element value is 0, s j The score value of the j-th input time frame of the initial Mel spectrum. The value of i is between 0 and t-1, and the value of j is between 0 and T-1.

[0070] By calculating the cumulative sum of scores To determine which output time frame each input time frame will map to, the elements W of the weight matrix... i,j Calculated using the following formula:

[0071]

[0072] This embodiment proposes a novel method for optimizing Mel feature processing. By compressing audio frames at an appropriate ratio, Mel features are simplified, effectively reducing the computational burden on the model. This method not only preserves key audio information but also significantly improves computational efficiency, helping the model achieve better performance in resource-constrained environments. Figure 3This is a schematic diagram of the Mel spectrum feature simplification process in an embodiment of the invention, including: acquiring the audio signal of the original waveform, extracting the Mel spectrum features of the audio signal into a Mel spectrum X of shape F×T, calculating the score of each time frame in the Mel spectrum X to obtain the score vector s, which is a 1*T matrix, calculating the weight matrix W based on the score vector s, the weight matrix W having a shape of t×T, and finally merging and multiplying the weight matrix W and the Mel spectrum X to obtain a simplified Mel spectrum O of shape F×t.

[0073] The scheme in this embodiment implements a method for filtering and retaining audio frames with obvious features by calculating the importance score of audio frames, thereby simplifying the features in the audio signal. The simplified Mel spectrum generated in this way retains the key information with high importance in the audio while reducing redundant data. By selecting audio frames with high importance scores, this embodiment not only reduces the amount of computation but also improves the accuracy and generalization ability of the model, making it suitable for improving the efficiency and performance of audio analysis tasks. By evaluating the importance score of audio frames, useless information in the Mel spectrum is compressed, reducing the computational cost of the model and improving the detection efficiency of the model.

[0074] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0075] Example 2

[0076] This embodiment also provides an audio signal feature extraction device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0077] Figure 4 This is a structural block diagram of an audio signal feature extraction device according to an embodiment of the present invention, such as... Figure 4 As shown, the device includes:

[0078] Acquisition module 40 is used to acquire audio signals;

[0079] Extraction module 42 is used to extract the initial Mel spectrum of the audio signal;

[0080] Calculation module 44 is used to calculate the weight matrix of the initial Mel spectrum, wherein the weight matrix is ​​used to compress the number of time frames in the initial Mel spectrum;

[0081] The operation module 46 is used to perform matrix multiplication on the weight matrix and the initial Mel spectrum to obtain the simplified Mel spectrum.

[0082] Optionally, the extraction module includes: a transformation unit for performing a short-time Fourier transform on the audio signal to obtain time-frequency features; a filtering unit for filtering the time-frequency features using a Mel filter bank to obtain spectral energy; and a processing unit for taking the logarithm of the spectral energy to obtain an initial Mel spectrum.

[0083] Optionally, the calculation module includes: a calculation unit for calculating the score value of each input time frame in the initial Mel spectrum, wherein the score value is used to characterize the importance of the corresponding input time frame, the initial Mel spectrum includes multiple input time frames, and the score value is between 0 and 1; and a generation unit for generating a weight matrix based on the score values ​​of all input time frames.

[0084] Optionally, the computation unit includes: a convolution subunit, configured to perform a one-dimensional convolution on each input time frame in the initial Mel spectrum to obtain the feature value of the time frame in the time dimension; a generation subunit, configured to generate a 1×T first score vector based on the feature values ​​of all time frames, where T is the number of input time frames in the initial Mel spectrum; an acquisition subunit, configured to acquire the number t of output time frames of the simplified Mel spectrum; and a processing subunit, configured to normalize the first score vector according to the number t to obtain a second score vector of the initial Mel spectrum, wherein the second score vector includes the score value of each input time frame in the initial Mel spectrum.

[0085] Optionally, the processing subunit is further configured to: normalize the first score vector using the following formula to obtain the second score vector s of the initial Mel spectrum: Among them, s ′ Let s be the first score vector. ′ k The value of the input time frame k in the first score vector.

[0086] Optionally, the generation unit includes: a construction subunit for constructing a t×T empty matrix, where T is the number of input time frames in the initial Mel spectrum and t is the number of output time frames in the simplified Mel spectrum; a calculation subunit for calculating the cumulative sum of the scores of all input time frames; and a processing subunit for calculating W for each element W in the empty matrix based on the cumulative sum. i,j The element values ​​are used to obtain the weight matrix, where the element W... i,j This represents the contribution of the j-th input time frame of the initial Mel spectrum to the i-th output time frame of the simplified Mel spectrum.

[0087] Optionally, the processing subunit is further configured to: for each element W i,j Calculate the score of the initial Mel spectrum from the first input time frame to the j-th input time frame. judge Whether it is true; if Establishment, determination of element W i,j The element value is s j ;like This is not true; determine element W. i,j The element value is 0, s j The score value for the j-th input time frame of the initial Mel spectrum.

[0088] It is not difficult to see that this embodiment is a device embodiment corresponding to the above method embodiment, and this embodiment can be implemented in conjunction with the above method embodiment. The relevant technical details mentioned in the above method embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Correspondingly, the relevant technical details mentioned in this embodiment can also be applied to the above method embodiment. It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but is not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0089] Example 3

[0090] This application also provides an electronic device. Figure 5 This is a structural diagram of an electronic device according to an embodiment of the present invention, such as... Figure 5 As shown, it includes a processor 51, a communication interface 52, a memory 53, and a communication bus 54. The processor 51, the communication interface 52, and the memory 53 communicate with each other through the communication bus 54. The memory 53 is used to store computer programs.

[0091] When the processor 51 executes the program stored in the memory 53, it performs the following steps: acquiring an audio signal; extracting the initial Mel spectrum of the audio signal; calculating the weight matrix of the initial Mel spectrum, wherein the weight matrix is ​​used to compress the number of time frames in the initial Mel spectrum; and performing matrix multiplication on the weight matrix and the initial Mel spectrum to obtain the simplified Mel spectrum.

[0092] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0093] The communication interface is used for communication between the aforementioned terminal and other devices.

[0094] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0095] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0096] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the audio signal feature extraction methods described in the above embodiments.

[0097] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the audio signal feature extraction methods in the above embodiments.

[0098] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0099] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0100] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0102] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0103] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0105] The above description is only a preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for feature extraction of an audio signal, characterized in that, include: Acquire audio signals; Extract the initial Mel spectrum of the audio signal; Calculate the weight matrix of the initial Mel spectrum, wherein the weight matrix is ​​used to compress the number of time frames in the initial Mel spectrum; The simplified Mel spectrum is obtained by performing matrix multiplication on the weight matrix and the initial Mel spectrum.

2. The method according to claim 1, characterized in that, Extracting the initial Mel spectrum of the audio signal includes: Perform a short-time Fourier transform on the audio signal to obtain its time-frequency characteristics; The time-frequency characteristics are filtered using a Mel filter bank to obtain the spectral energy; The initial Mel spectrum is obtained by taking the logarithm of the spectral energy.

3. The method according to claim 1, characterized in that, Calculating the weight matrix of the initial Mel spectrum includes: Calculate the score value for each input time frame in the initial Mel spectrum, wherein the score value is used to characterize the importance of the corresponding input time frame, the initial Mel spectrum includes multiple input time frames, and the score value takes a value between 0 and 1; A weight matrix is ​​generated based on the scores of all input time frames.

4. The method according to claim 3, characterized in that, Calculating the score value for each input time frame in the initial Mel spectrum includes: For each input time frame in the initial Mel spectrum, a one-dimensional convolution is performed on the input time frame to obtain the feature value of the time frame in the time dimension; A first score vector of 1×T is generated based on the feature values ​​of all time frames, where T is the number of input time frames in the initial Mel spectrum; The number of output time frames t of the simplified Mel spectrum is obtained; The first score vector is normalized according to the quantity t to obtain the second score vector of the initial Mel spectrum, wherein the second score vector includes the score value of each input time frame in the initial Mel spectrum.

5. The method according to claim 4, characterized in that, The first score vector is normalized according to the quantity t to obtain the second score vector of the initial Mel spectrum, including: The first score vector is normalized using the following formula to obtain the second score vector s of the initial Mel spectrum: Where s′ is the first score vector, s′ k The value of the input time frame k in the first score vector.

6. The method according to claim 3, characterized in that, The weight matrix generated based on the scores of all input time frames includes: Construct an empty matrix t×T, where T is the number of input time frames in the initial Mel spectrum and t is the number of output time frames in the simplified Mel spectrum; Calculate the cumulative sum of scores for all input time frames; Based on the cumulative sum, calculate W for each element in the empty matrix. i,j The element values ​​are used to obtain the weight matrix, where the element W... i,j This represents the contribution of the j-th input time frame of the initial Mel spectrum to the i-th output time frame of the simplified Mel spectrum.

7. The method according to claim 6, characterized in that, Based on the cumulative sum, calculate W for each element in the empty matrix. i,j The element values ​​include: For each element W i，j Calculate the score of the initial Mel spectrum from the first input time frame to the j-th input time frame. judge Is it valid? like Establishment, determination of element W i,j The element value is s j If i≤ This is not true; determine element W. i,j The element value is 0, s j The score value for the j-th input time frame of the initial Mel spectrum.

8. A feature extraction device for audio signals, characterized in that, include: The acquisition module is used to acquire audio signals; An extraction module is used to extract the initial Mel spectrum of the audio signal; A calculation module is used to calculate the weight matrix of the initial Mel spectrum, wherein the weight matrix is ​​used to compress the number of time frames in the initial Mel spectrum; The computation module is used to perform matrix multiplication on the weight matrix and the initial Mel spectrum to obtain a simplified Mel spectrum.

9. A storage medium, characterized in that, The storage medium stores a computer program, wherein the computer program is configured to execute the method described in any one of claims 1 to 7 when it is run.

10. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the method as described in any one of claims 1 to 7.

Images