A learnable time-frequency transformation method

Abstract

The present invention relates to a learnable time-frequency transformation method, comprising a data collector and a computer system connected in sequence, wherein a learnable time-frequency transformation operation unit is provided in the computer; the window parameters of the learnable time-frequency transformation are updated by the gradient obtained by the deep neural network back-propagation algorithm. The trained learnable time-frequency transformation operation unit can be accelerated by the fast Fourier transform algorithm to improve the execution efficiency. The window parameters of the learnable time-frequency transformation are updated by the back-propagation algorithm according to the set loss function, thereby eliminating the steps of manual design, selection, parameter adjustment, etc. of the window function or wavelet basis in the traditional time-frequency transformation. In addition, after the training of the learnable time-frequency transformation is completed, the parameters are fixed, becoming a new time-frequency transformation, and the optimal time-frequency characteristics for the target task can be obtained.

Description

Technical Field

[0001] The invention relates to a novel learnable time-frequency transformation method, belonging to the technical field of signal processing. Background Art

[0002] In the field of signal processing, time-frequency transformation of time series signals can obtain the time-frequency distribution of time domain signals, and then analyze the change characteristics of time domain signals in time domain and frequency domain. It is a commonly used signal representation method in the field of signal processing. Time-frequency transformation technology has a wide range of applications and has been applied to the analysis of various time series signals, such as speech signals, music signals, medical electrophysiological signals, etc. Common time-frequency transformation methods include short-time Fourier transform and continuous wavelet transform. Although these time-frequency transformation methods can effectively express the time-frequency characteristics of signals to a certain extent, hyperparameters such as window functions and wavelet bases still need to be manually selected for different signal samples. For example, short-time Fourier transform requires manual selection of window functions such as rectangular windows and Hanning windows; continuous wavelet transform requires manual selection of mother wavelets. These parameters can greatly affect the expression ability of time-frequency transformation. In addition, the selection and optimization of these hyperparameters in current time-frequency methods have always relied on experience selection, and have not been linked to performance indicators of classification tasks such as accuracy to achieve automatic parameter optimization. In 1996, RG Stockwell et al. published a paper titled "Localization of the complex spectrum: the S transform" in IEEE Transactions on Signal Processing (Volume 44, Issue 4). They proposed a time-frequency transform that combines the rapidity of the short-time Fourier transform with the multi-resolution characteristics of the continuous wavelet transform and named it the S transform. This method has been widely used in the field of signal processing. However, this method requires manual selection of appropriate window parameters for different signals to obtain the optimal time-frequency resolution, and cannot adaptively adjust the window parameters, making it difficult to obtain the optimal time-frequency transform of the signal. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention proposes a learnable time-frequency transformation method. The preset window parameters of the traditional time-frequency transformation are replaced with learnable parameters, and the optimal time-frequency transformation of the signal for the target task can be obtained by updating the back-propagation algorithm.

[0004] The technical solution of the present invention is as follows:

[0005] A learnable time-frequency conversion method is completed by the following system, which includes a data collector and a computer connected in sequence, wherein a learnable time-frequency conversion operation unit is provided in the computer; the data collector is used to convert an analog signal into a digital signal and transmit it to the computer, and the learnable time-frequency conversion operation unit in the computer obtains a time-frequency spectrum of the signal, and the specific steps of the method are as follows:

[0006] 1) Collect training data

[0007] For signal x i (t), i=1,2,…N d , obtain N through the data logger d A time series sample x i [kT] is used as training data, y i ∈{1,2,...,N c} is the category corresponding to each sample, where k = 1, 2, ..., N s is the time series index, N s is the number of sampling points, N c is the total number of sample categories, Fs is the sampling frequency, and T = 1 / Fs is the sampling period;

[0008] 2) Training the learnable time-frequency transformation operation unit

[0009] First, initialize the learnable complex window parameter w τ,f ;w τ,f The imaginary parts of are all initialized to zero, and the real parts are initialized to the Gaussian window function, that is:

[0010]

[0011] where τ∈[T,N s T] is the time delay variable, f∈[0,Fs / 2-1] is the frequency variable; then, the learnable weight matrix P is initialized; the parameters in the learnable weight matrix P are initialized to random numbers that obey a Gaussian distribution with zero mean and unit standard deviation;

[0012] Train the learnable parameters w of the learnable time-frequency transform τ,f and P; v represents the number of iterations, starting from v = 1 and performing the following iteration cycle:

[0013] a) For the input x i (t), which can learn the time-frequency transform Q i (τ,f) is expressed as:

[0014]

[0015] Where * is the convolution operator, which is simplified and accelerated by Fourier transform. Fourier transform both sides of equation (II) to obtain:

[0016]

[0017] Where X i (α) is the time series x i The Fourier transform result of (τ) is is the Fourier transform operator that converts the time delay variable τ into the frequency Fourier variable α; taking the inverse Fourier transform on both sides of the equation, we can get:

[0018]

[0019] in is the inverse Fourier transform operator that converts the frequency Fourier variable α into the time delay variable τ, is the complex window w τ,f The Fourier transform result of the signal x is obtained i (t) is a learnable time-frequency transform Q i (τ,f); then according to the convolution theorem, the discretized learnable time-frequency transform spectrum can be expressed as:

[0020]

[0021] in For x i [kT] The discrete Fourier transform result at frequency, is the complex window w τ,f The discrete Fourier transform result of p=1,2,…,N s is the time index variable, m=0,1,…,(Fs / 2-1)N s T is the frequency index variable, n=-Ns / 2,-Ns / 2+1,…,Ns / 2-1 is the frequency Fourier index variable;

[0022] b) According to formula (VI), the learnable time-frequency transform spectrogram matrix S is obtained i [pT,m / N s T]:

[0023]

[0024] Where conj(·) represents the complex conjugate operation function, ⊙ represents the Hadamard product;

[0025] c) The learnable time-frequency transform spectrogram matrix S i [pT,m / N s T] is expanded into a one-dimensional eigenvector Z i, and then through the learnable

[0026] The weight matrix P is mapped to have N c The eigenvector R of the elements i :

[0027] R i =Z i ×P (VII)

[0028] d) According to the feature vector R output in c) i , the current loss value is calculated through the loss function J(θ); the loss function

[0029] J(θ) is the cross entropy loss function:

[0030]

[0031] where θ represents all learnable parameters (i.e. w τ,f and P), R related to θ i (j) represents the eigenvector R i The j-th element value of i is the true sample category of the i-th time series sample, and 1{·} is the representative function;

[0032] e) Based on the loss value calculated in d), all learnable parameters are updated according to formula (IX):

[0033] θ v+1 =θ v -μ▽J(θ v )+λ(θ v -θ v-1 ) (IX)

[0034] Where μ is the learning rate, λ is the momentum parameter, which is used to adjust the influence of the gradient generated by the previous iteration on the gradient of the current iteration, and θ v represents all learnable parameters at the vth iteration, ▽J(θ v ) is the loss function J(θ v ) for θ v The gradient value of

[0035] f) If v = N e , where N e is the maximum number of iterations, the time-frequency transform operation unit can be trained and the trained complex window parameters can be obtained. Then jump out of the loop and go to step 3); otherwise, add 1 to v and return to step a);

[0036] 3) Collect data to be analyzed

[0037] Signal to be analyzed Obtaining time series samples through a data collector As the data to be analyzed, k = 1, 2, ..., N s , N s is the number of sampling points, Fs is the sampling frequency, and T=1 / Fs is the sampling period;

[0038] 4) Input the trained learnable time-frequency transform operation unit for calculation

[0039] The time series data to be analyzed Send it to the trained learnable time-frequency transform unit for calculation, and then the learnable time-frequency transform spectrum can be calculated.

[0040]

[0041] in for The discrete-time Fourier transform of p = 1, 2, ..., N s is the time index variable, m=0,1,…,(Fs / 2-1)N s T is the frequency index variable, n = -N s / 2,-N s / 2+1,…,N s / 2-1 is the frequency Fourier index variable, For complex windows The Fourier transform result of

[0042] 5) Output learnable time-frequency transform spectrogram

[0043]

[0044] Where conj(·) represents the complex conjugate operation function, and ⊙ represents the Hadamard product.

[0045] The beneficial effects of the present invention are:

[0046] All complex window parameters in the learnable time-frequency transform are automatically updated through the back-propagation algorithm, eliminating the steps of manually designing, selecting, and adjusting parameters of window functions or wavelet bases in traditional time-frequency transforms. After the training of the learnable time-frequency transform is completed, the parameters are fixed, becoming a new type of time-frequency transform that can obtain the optimal time-frequency features for the target task. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a flowchart of the steps of the learnable time-frequency conversion method of the present invention;

[0048] Figure 2 A flowchart of the steps for training a learnable time-frequency transform unit;

[0049] Figure 3 is the real part of the initial value of the window parameter of the learnable time-frequency transform;

[0050] Figure 4(a) shows the real part of the window parameters after the learnable time-frequency transform is trained;

[0051] Figure 4(b) shows the imaginary part of the window parameters after the learnable time-frequency transform is trained;

[0052] Figure 5 It is a time domain waveform diagram of a time series to be analyzed;

[0053] Figure 6 is obtained by learnable time-frequency transformation Figure 5 Spectrogram of the time series to be analyzed. DETAILED DESCRIPTION

[0054] The present invention is further described below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited thereto;

[0055] Example

[0056] A learnable time-frequency conversion method is implemented by the following system, which includes a data collector and a computer connected in sequence, wherein the computer is provided with a learnable time-frequency conversion operation unit; the data collector is used to convert an analog signal into a digital signal and transmit it to the computer, and the learnable time-frequency conversion operation unit in the computer obtains a time-frequency spectrum of the signal, such as Figure 1 As shown, the specific steps of the method are as follows:

[0057] 1) Collect training data

[0058] For signal x i (t), i=1,2,…N d , obtain N through the data logger d = 128 time series samples x i [kT] is used as training data, y i ∈{1,2,...,N c} is the category corresponding to each sample, where k = 1, 2, ..., N s is the time series index, N s =256 is the number of sampling points, N c =2 is the total number of sample categories, Fs=256Hz is the sampling frequency, and T=1 / Fs=0.0039s is the sampling period.

[0059] 2) Train the learnable time-frequency transform operation unit (the steps are as follows Figure 2 (shown)

[0060] First, initialize the learnable complex window parameter wτ,f ;w τ,f The imaginary parts of are all initialized to zero, and the real parts are initialized to the Gaussian window function, such as Figure 3 As shown, that is:

[0061]

[0062] Where τ∈[0.0039,1] is the time delay variable, f∈[0,127] is the frequency variable; the parameters in the learnable weight matrix P are initialized to random numbers that obey a Gaussian distribution with zero mean and unit standard deviation;

[0063] Train the learnable parameters w of the learnable time-frequency transform τ,f and P; v represents the number of iterations, starting from v = 1 and performing the following iteration cycle:

[0064] a) For the input x i (t), which can learn the time-frequency transform Q i (τ,f) is expressed as:

[0065]

[0066] Where * is the convolution operator, which is simplified and accelerated by Fourier transform. Fourier transform both sides of equation (II) to obtain:

[0067]

[0068] Where X i (α) is the time series x i The Fourier transform result of (τ) is is the Fourier transform operator that converts the time delay variable τ into the frequency Fourier variable α; taking the inverse Fourier transform on both sides of the equation, we can get:

[0069]

[0070] in is the inverse Fourier transform operator that converts the frequency Fourier variable α into the time delay variable τ, is the complex window w τ,f The Fourier transform result of the signal x is obtained i (t) is a learnable time-frequency transform Q i (τ,f); then according to the convolution theorem, the discretized learnable time-frequency transform spectrum can be expressed as:

[0071]

[0072] in For x i [kT] The discrete Fourier transform result at frequency, is the complex window w τ,f The discrete Fourier transform result of , p = 1, 2, ..., 256 is the time index variable, m = 0, 1, ..., 127 is the frequency index variable, n = -128, -127, ..., 127 is the frequency Fourier index variable;

[0073] b) According to formula (VI), the learnable time-frequency transform spectrogram matrix S is obtained i [pT,m / N s T]:

[0074]

[0075] Where conj(·) represents the complex conjugate operation function, ⊙ represents the Hadamard product;

[0076] c) The learnable time-frequency transform spectrogram matrix S obtained in step b) is transformed i [pT,m / N s T] is expanded into a one-dimensional eigenvector Then the learnable weight matrix Mapped to a eigenvector R with 2 elements i :

[0077] R i =Z i ×P (VII)

[0078] d) According to the feature vector R output in c) i , the current loss value is calculated through the loss function J(θ); the network loss function J(θ) is the cross entropy loss function:

[0079]

[0080] Where θ represents all learnable parameters, and R associated with θ i (j) represents the eigenvector R i The j-th element value of i is the true sample category of the i-th time series sample, and 1{·} is the representative function;

[0081] e) Based on the loss value calculated in d), all learnable parameters are updated according to formula (IX):

[0082] θ v+1 =θ v -μ▽J(θ v )+λ(θ v -θ v-1 ) (IX)

[0083] Where μ = 0.001 is the learning rate, λ = 0.9 is the momentum parameter, which is used to adjust the influence of the gradient generated by the previous iteration on the gradient of the current iteration, and θ v Represents all learnable parameters at the vth iteration (i.e., w τ,f and P), ▽J(θ v ) is the loss function J(θ v ) for θ v The gradient value of .

[0084] f) If v = 200, the time-frequency transform operation unit can be trained and the trained complex window parameters can be obtained. (like Figure 4(a) and 4(b) As shown), then jump out of the loop and go to step 3); otherwise, let v increase by 1 and return to step a);

[0085] 3) Collect data to be analyzed

[0086] Signal to be analyzed Obtaining time series samples through a data collector As the data to be analyzed, Figure 5 As shown, where k = 1, 2, ..., N s , N s =256 is the number of sampling points, Fs=256Hz is the sampling frequency, T=1 / Fs=0.0039s is the sampling period;

[0087] 4) Input the trained learnable time-frequency transform operation unit for calculation

[0088] The time series data to be analyzed Send it to the trained learnable time-frequency transform unit for calculation, and then the learnable time-frequency transform spectrum can be calculated.

[0089]

[0090] in for The discrete time Fourier transform of p = 1, 2, ..., 256 is the time index variable, m = 0, 1, ..., 127 is the frequency index variable, n = -128, -127, ..., 127 is the frequency Fourier index variable, For complex windows The Fourier transform result of

[0091] 5) Output learnable time-frequency transform spectrogram

[0092]

[0093] Where conj(·) represents the complex conjugate operation function, ⊙ represents the Hadamard product; the output learnable time-frequency transform spectrum is as follows Figure 6 shown.

Claims

1. A learnable time-frequency conversion method is completed by the following system, which includes a data collector and a computer connected in sequence, wherein the computer is provided with a learnable time-frequency conversion operation unit; the data collector is used to convert an analog signal into a digital signal and transmit it to the computer, and the learnable time-frequency conversion operation unit in the computer obtains a time-frequency spectrum of the signal, and the specific steps of the method are as follows: 1) Collect training data For signal x i (t), i=1,2,…N d , obtain N through the data logger d Time series samples x i [kT] is used as training data, y i ∈{1,2,...,N c } is the category corresponding to each sample, where k=1,2,…,N s is the time series index, N s is the number of sampling points, N c is the total number of sample categories, Fs is the sampling frequency, and T = 1 / Fs is the sampling period; 2) Training the learnable time-frequency transformation operation unit First, initialize the learnable complex window parameter w τ,f ;w τ,f The imaginary parts of are all initialized to zero, and the real parts are initialized to the Gaussian window function, that is: where τ∈[T,N s T] is the time delay variable, f∈[0,Fs / 2-1] is the frequency variable; then, the learnable weight matrix P is initialized; the parameters in the learnable weight matrix P are initialized to random numbers that obey a Gaussian distribution with zero mean and unit standard deviation; Train the learnable parameters w of the learnable time-frequency transform τ,f and P; Let v represent the number of iterations, and start the following iteration cycle from v=1: a) For the input x i (t), which can learn the time-frequency transform Q i (τ,f) is expressed as: Where * is the convolution operator, which is simplified and accelerated by Fourier transform. Fourier transform both sides of equation (II) to obtain: Where X i (α) is the time series x i The Fourier transform result of (τ) is is the Fourier transform operator that converts the time delay variable τ into the frequency Fourier variable α; taking the inverse Fourier transform on both sides of the equation, we can get: in is the inverse Fourier transform operator that converts the frequency Fourier variable α into the time delay variable τ, is the complex window w τ,f The Fourier transform result of the signal x is obtained i (t) is a learnable time-frequency transform Q i (τ,f); then according to the convolution theorem, the discretized learnable time-frequency transform spectrum can be expressed as: in For x i [kT] The discrete Fourier transform result at frequency, is the complex window w τ,f The discrete Fourier transform result of p=1,2,…,N s is the time index variable, m=0,1,…,(Fs / 2-1)N s T is the frequency index variable, n = -N s / 2,-N s / 2+1,…,N s / 2-1 is the frequency Fourier index variable; b) According to formula (VI), the learnable time-frequency transform spectrogram matrix S is obtained i [pT,m / N s T]: Where conj(·) represents the complex conjugate operation function, and ⊙ represents the Hadamard product; c) The learnable time-frequency transform spectrogram matrix S i [pT,m / N s T] is expanded into a one-dimensional eigenvector Z i , and then mapped to N by the learnable weight matrix P c The eigenvector R of the elements i : R i =Z i ×P (VII) d) According to the feature vector R output in c) i , the current loss value is calculated through the loss function J(θ); the loss function J(θ) is the cross entropy loss function: Where θ represents all learnable parameters, and R associated with θ i (j) represents the eigenvector R i The j-th element value of i is the true sample category of the i-th time series sample, and 1{·} is the representative function; e) Based on the loss value calculated in d), all learnable parameters are updated according to formula (IX): Where μ is the learning rate, λ is the momentum parameter, which is used to adjust the influence of the gradient generated by the previous iteration on the gradient of the current iteration, and θ v represents all learnable parameters at the vth iteration, is the loss function J(θ v ) for θ v The gradient value of f) If v = N e , where N e is the maximum number of iterations, the time-frequency transform operation unit can be trained and the trained complex window parameters can be obtained. Then jump out of the loop and go to step 3); otherwise, add 1 to v and return to step a); 3) Collect data to be analyzed Signal to be analyzed Obtaining time series samples through a data collector As the data to be analyzed, k = 1, 2, ..., N s , N s is the number of sampling points, Fs is the sampling frequency, and T=1 / Fs is the sampling period; 4) Input the trained learnable time-frequency transform operation unit for calculation The time series data to be analyzed Send it to the trained learnable time-frequency transform unit for calculation, and then the learnable time-frequency transform spectrum can be calculated. in for The discrete-time Fourier transform of p = 1, 2, ..., N s is the time index variable, m=0,1,…,(Fs / 2-1)N s T is the frequency index variable, n = -N s / 2,-N s / 2+1,…,N s / 2-1 is the frequency Fourier index variable, For complex windows The Fourier transform result of 5) Output learnable time-frequency transform spectrogram Where conj(·) represents the complex conjugate operation function, and ⊙ represents the Hadamard product.

Images