Complex time-frequency spectrogram enhancement method and system

Abstract

The invention discloses a complex time-frequency spectrogram enhancement method. The method comprises the following steps: constructing a random signal database based on frequency modulation signals; performing discrete short-time Fourier transform on the signal to obtain a complex time-frequency spectrogram data set; constructing a complex number time-frequency spectrogram enhancement primary model based on complex number convolution and complex number deconvolution, and training to obtain a complex number time-frequency spectrogram enhancement model; and enhancing the actual complex number time-frequency spectrogram by adopting the complex number time-frequency spectrogram enhancement model. The invention also discloses a system for realizing the method for enhancing the complex number time-frequency spectrogram. According to the method, the complex number time-frequency spectrogram enhancement model based on the complex number convolutional neural network is designed and trained, so that enhancement of the complex number time-frequency spectrogram can be realized, the reliability is higher, and the accuracy is better.

Description

Technical Field

[0001] The present invention belongs to the technical field of signal processing, and in particular relates to a complex time-frequency spectrum enhancement method and system. Background Art

[0002] In the process of signal processing, short-time Fourier transform is one of the most commonly used time-frequency analysis methods, which can convert a one-dimensional complex signal into a two-dimensional complex time-frequency spectrum. However, the short-time Fourier transform uses a window function to segment the signal, and the truncated signal will produce frequency leakage; at the same time, the introduction of the window function also makes this method subject to the limitation of the Heisenberg uncertainty criterion. Therefore, the complex time-frequency spectrum obtained by the short-time Fourier transform scheme generally has a low time-frequency resolution and erroneous time-frequency information. It is precisely because of this kind of reason that when the short-time Fourier transform scheme is applied in precision fields (such as through-wall radar trajectory tracking, etc.), the accuracy and reliability of the complex time-frequency spectrum obtained are poor. Summary of the invention

[0003] One of the objectives of the present invention is to provide a complex time-frequency spectrum enhancement method with high reliability and good accuracy.

[0004] A second objective of the present invention is to provide a system for implementing the complex time-frequency spectrum enhancement method.

[0005] The complex time-frequency spectrum enhancement method provided by the present invention comprises the following steps:

[0006] S1. Construct a random signal database based on FM signals;

[0007] S2. Performing a discrete short-time Fourier transform on the signal in the random signal database constructed in step S1 to obtain a complex time-frequency spectrum data set;

[0008] S3. Based on complex convolution and complex deconvolution, a complex time-frequency spectrum enhancement primary model is constructed;

[0009] S4. Using the complex time-frequency spectrum data set obtained in step S2, the complex time-frequency spectrum enhancement primary model constructed in step S3 is trained to obtain a complex time-frequency spectrum enhancement model;

[0010] S5. Use the complex time-frequency spectrum enhancement model obtained in step S4 to enhance the actual complex time-frequency spectrum.

[0011] The step S1 of constructing a random signal database based on the frequency modulation signal specifically includes the following steps:

[0012] Based on the cosine frequency modulation signal, construct the first random signal S 1 (k)

[0013]

[0014] Where M is the number of sub-signals; A m is the amplitude value of the mth sub-signal; B m is the first frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-100,100); C m is the second frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (0.001,10); D m is the third frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-8,8); E m is the fourth frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-5,5); θ m is the initial phase of the mth sub-signal and obeys the uniform distribution of (0,2π); k is time;

[0015] Based on the four-frequency modulation signal, construct the second random signal S 2 (k)

[0016]

[0017] Where F m is the fifth frequency adjustment coefficient of the mth sub-signal and is uniformly distributed in the range of (-50, 50); G m is the sixth frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-25, 25);

[0018] Randomly generate X groups of first random signals S 1 (k) and Y groups of second random signals S 2 (k);

[0019] Gaussian white noise is added to all randomly generated signals, and the signal-to-noise ratio of the signal after adding Gaussian white noise is guaranteed to be -5dB to 40dB.

[0020] Step S2 performs a discrete short-time Fourier transform on the signal in the random signal database constructed in step S1 to obtain a complex time-frequency spectrum data set, which specifically includes the following steps:

[0021] Performing a discrete short-time Fourier transform on the signal in the random signal database constructed in step S1;

[0022] The discrete short-time Fourier transform is performed using the following formula:

[0023]

[0024] Where N is the number of discrete short-time Fourier transform points; n is time; k is the frequency point; s(m) is the input signal; w(mn) is the set window function;

[0025] Finally, the data obtained after discrete short-time Fourier transform is normalized to obtain a complex time-frequency spectrum data set.

[0026] The step S3 described in which the complex time spectrum enhancement primary model is constructed based on complex convolution and complex deconvolution specifically includes the following steps:

[0027] The constructed complex time-frequency spectrogram enhancement primary model includes 10 two-dimensional complex convolutional layers with ReLu activation function and 10 two-dimensional complex deconvolutional layers with ReLu activation function;

[0028] A 2D complex convolutional layer with ReLu activation function is used to extract spectral information in the spectrogram and reduce the dimensionality of the input data;

[0029] A 2D complex deconvolution layer with ReLu activation function is used to recover the lost information in the spectrogram and enhance the spectrogram;

[0030] The first two-dimensional complex convolution layer with ReLu activation function to the tenth two-dimensional complex convolution layer with ReLu activation function are connected in series;

[0031] The output of the 10th 2D complex convolutional layer with ReLu activation function is directly connected to the input of the 1st 2D complex deconvolutional layer with ReLu activation function;

[0032] The 1st to 10th two-dimensional complex deconvolution layers with ReLu activation function are connected in series in sequence; in addition, the output of the i-th two-dimensional complex convolution layer with ReLu activation function is simultaneously connected to the input of the 11-i-th two-dimensional complex deconvolution layers with ReLu activation function; the value of i is 1 to 9; the number of output channels of the 10th two-dimensional complex deconvolution layer with ReLu activation function is 1.

[0033] The training described in step S4 specifically includes the following steps:

[0034] Construct loss function;

[0035] During the training process, back propagation is performed to update the constructed model until the model converges; the model converges, specifically, the value of the constructed loss function remains unchanged within a set range.

[0036] The step S5 adopts the complex time-frequency spectrum enhancement model obtained in step S4 to perform actual complex time-frequency spectrum enhancement, which specifically includes the following steps:

[0037] Acquire a complex signal of a time-spectrogram to be enhanced;

[0038] Performing discrete short-time Fourier transform on the complex signal of the acquired time-frequency spectrum to be enhanced to obtain a corresponding complex time-frequency spectrum;

[0039] Normalize the obtained complex time-frequency spectrum;

[0040] The normalized complex time-spectrum graph is input into the obtained complex time-spectrum graph enhancement model, and the output of the complex time-spectrum graph enhancement model is used as the enhanced complex time-spectrum graph to complete the actual enhancement of the complex time-spectrum graph.

[0041] The present invention also provides a system for implementing the complex time-frequency spectrum enhancement method, comprising a signal construction module, a signal processing module, a model construction module, a model training module and a data enhancement module; the signal construction module, the signal processing module, the model construction module, the model training module and the data enhancement module are connected in series in sequence; the signal construction module is used to construct a random signal database based on a frequency modulation signal, and upload the data information to the signal processing module; the signal processing module is used to perform a discrete short-time Fourier transform on the signal in the constructed random signal database according to the received data information to obtain a complex time-frequency spectrum data set, and upload the data information to the model construction module; the model construction module is used to construct a complex time-frequency spectrum enhancement primary model based on complex convolution and complex deconvolution according to the received data information, and upload the data information to the model training module; the model training module is used to train the complex time-frequency spectrum enhancement primary model constructed in step S3 according to the received data information, using the obtained complex time-frequency spectrum data set to obtain a complex time-frequency spectrum enhancement model, and upload the data information to the data enhancement module; the data enhancement module is used to perform actual complex time-frequency spectrum enhancement according to the received data information using the obtained complex time-frequency spectrum enhancement model.

[0042] The complex time-frequency spectrum enhancement method and system provided by the present invention designs and trains a complex time-frequency spectrum enhancement model based on a complex convolutional neural network. Therefore, the present invention can not only enhance the complex time-frequency spectrum, but also has higher reliability and better accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The figure is a schematic diagram of the method flow of the present invention.

[0044] Figure 2 Schematic diagram of the functional modules of the system of the present invention. DETAILED DESCRIPTION

[0045] like Figure 1 The method flow chart of the method of the present invention is shown as follows: the complex time-frequency spectrum enhancement method disclosed in the present invention comprises the following steps:

[0046] S1. Based on the frequency modulation signal, a random signal database is constructed; specifically, the steps include:

[0047] Based on the cosine frequency modulation signal, construct the first random signal S 1 (k)

[0048]

[0049] Where M is the number of sub-signals; A m is the amplitude value of the mth sub-signal; B m is the first frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-100,100); C m is the second frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (0.001,10); D m is the third frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-8,8); E m is the fourth frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-5,5); θ m is the initial phase of the mth sub-signal and obeys the uniform distribution of (0,2π); k is time;

[0050] Based on the four-frequency modulation signal, construct the second random signal S 2 (k)

[0051]

[0052] Where F m is the fifth frequency adjustment coefficient of the mth sub-signal and is uniformly distributed in the range of (-50, 50); G m is the sixth frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-25, 25);

[0053] Randomly generate X groups of first random signals S 1 (k) and Y groups of second random signals S 2 (k);

[0054] Gaussian white noise is added to all randomly generated signals, and the signal-to-noise ratio of the signal after adding Gaussian white noise is guaranteed to be -5dB to 40dB;

[0055] S2. Performing a discrete short-time Fourier transform on the signal in the random signal database constructed in step S1 to obtain a complex time-frequency spectrum data set; specifically comprising the following steps:

[0056] Performing a discrete short-time Fourier transform on the signal in the random signal database constructed in step S1;

[0057] The discrete short-time Fourier transform is performed using the following formula:

[0058]

[0059] Where N is the number of discrete short-time Fourier transform points; n is time; k is the frequency point; s(m) is the input signal; w(mn) is the set window function;

[0060] In the natural environment, the energy range of the signal is diverse, and normalization processing is essential to prevent the network from being affected by the change of input amplitude, thereby preventing the network from failing to converge; therefore, the data obtained after the discrete short-time Fourier transform is finally normalized to obtain a complex time-frequency spectrum data set; the preferred normalization processing scheme is the maximum amplitude value normalization processing scheme;

[0061] S3. Based on complex convolution and complex deconvolution, a primary model for complex time spectrum enhancement is constructed; specifically, the steps are as follows:

[0062] The constructed complex time-frequency spectrogram enhancement primary model includes 10 two-dimensional complex convolutional layers with ReLu activation function and 10 two-dimensional complex deconvolutional layers with ReLu activation function;

[0063] A 2D complex convolutional layer with ReLu activation function is used to extract spectral information in the spectrogram and reduce the dimensionality of the input data;

[0064] A 2D complex deconvolution layer with ReLu activation function is used to recover the lost information in the spectrogram and enhance the spectrogram;

[0065] The first two-dimensional complex convolution layer with ReLu activation function to the tenth two-dimensional complex convolution layer with ReLu activation function are connected in series;

[0066] The output of the 10th 2D complex convolutional layer with ReLu activation function is directly connected to the input of the 1st 2D complex deconvolutional layer with ReLu activation function;

[0067] The first two-dimensional complex deconvolution layer with ReLu activation function to the tenth two-dimensional complex deconvolution layer with ReLu activation function are connected in series in sequence; in addition, the output of the i-th two-dimensional complex convolution layer with ReLu activation function is simultaneously connected to the input of the 11-i-th two-dimensional complex deconvolution layer with ReLu activation function; the value of i is 1 to 9; the number of output channels of the tenth two-dimensional complex deconvolution layer with ReLu activation function is 1; this residual connection method between the convolution layer and the deconvolution layer can greatly increase the convergence speed and accuracy of the model;

[0068] S4. Using the complex time-frequency spectrum data set obtained in step S2, the complex time-frequency spectrum enhancement primary model constructed in step S3 is trained to obtain a complex time-frequency spectrum enhancement model;

[0069] The training specifically includes the following steps:

[0070] Construct loss function;

[0071] During the training process, back propagation is performed to update the constructed model until the model converges; the model converges, specifically, the value of the constructed loss function remains unchanged within a set range;

[0072] S5. Using the complex time-frequency spectrum enhancement model obtained in step S4, the actual complex time-frequency spectrum enhancement is performed; specifically comprising the following steps:

[0073] Acquire a complex signal of a time-spectrogram to be enhanced;

[0074] Performing discrete short-time Fourier transform on the complex signal of the acquired time-frequency spectrum to be enhanced to obtain a corresponding complex time-frequency spectrum;

[0075] Normalizing the obtained complex time-frequency spectrum (preferably also performing maximum value normalization processing);

[0076] The normalized complex time-spectrum graph is input into the obtained complex time-spectrum graph enhancement model, and the output of the complex time-spectrum graph enhancement model is used as the enhanced complex time-spectrum graph to complete the actual enhancement of the complex time-spectrum graph.

[0077] In order to effectively verify the time-frequency enhancement capability of the proposed AS-Net, three different indicators include: Rayleigh entropy (RE), l 1 Loss and energy correlation (EC) are used to quantify the performance of the proposed algorithm and other time-frequency analysis methods. RE usually evaluates the energy compression performance of the algorithm. The smaller the value, the stronger its energy compression ability. RE can be expressed by the formula as follows:

[0078]

[0079] Where TF[i,k] represents the time-frequency distribution of the signal after TFA, μ is a quantitative adjustment parameter, preferably set to 3; 1 The loss represents the difference between the time-frequency spectrogram predicted by time-frequency analysis and the ideal time-frequency spectrogram. 1 Distance, EC is used to evaluate the similarity between the time-frequency distribution and the ideal time-frequency distribution. The larger the EC, the more similar the predicted time-frequency distribution is to the ideal time-frequency distribution. It can be expressed by the following formula:

[0080]

[0081] Among them TF ref Indicates the ideal time-frequency distribution.

[0082] Two different simulation signals were used for verification, and several different algorithms including STFT, SST, SET, MSST, WVD, and ADTFD were compared to verify the superiority of the proposed algorithm. 1 (t), S 2 (t) can be expressed as follows:

[0083]

[0084] The duration of the above signals is set to 5.12s, and their sampling frequency is 100Hz. In order to prove the effectiveness of AS-Net in the case of noise, S 1 (t) and S 2 (t) are all additive white Gaussian noise with a signal-to-noise ratio of 10dB.

[0085] The STFT spectrogram has obvious frequency blurring due to its low time-frequency resolution. When the human body is engaged in activities, the rotation of multiple trunks will produce similar situations. STFT will introduce frequencies unrelated to movement, thereby affecting the accuracy of activity recognition. SST, SET and MSST effectively concentrate the energy of the spectrum by introducing synchronous compression, but frequency errors still exist. WVD, as a secondary time-frequency analysis method, performs Fourier transform on the autocorrelation function of the signal to give it the highest theoretical time-frequency resolution. However, its cross-term problem causes its spectrum to be obviously blurred under multi-component signals. ADTFD eliminates the influence of cross-terms to a certain extent by introducing kernel functions, but there are still some cross-terms that affect the spectrum. Compared with the above algorithms, the proposed AS-Net has the best performance. In order to quantify the performance capabilities of the proposed different algorithms and to effectively evaluate the three different indicators RE, l 1Loss,EC,All spectra are normalized to [0, 1], and the results are shown in Table 1.

[0086] Table 1 Index comparison diagram

[0087]

[0088] It can also be seen from Table 1 that the comprehensive performance of the method of the present invention is the best.

[0089] The complex time-frequency spectrum enhancement method provided by the present invention is particularly suitable for radar-based human activity perception and through-wall radar trajectory tracking.

[0090] When applied to human activity perception scenarios of radar, the action recognition method including the complex time-frequency spectrum enhancement method of the present invention specifically includes the following steps:

[0091] A. Obtain radar human activity monitoring signals to be enhanced;

[0092] B. Perform short-time Fourier transform of different window lengths on the monitoring signal obtained in step A to obtain time-frequency spectrograms with different window lengths

[0093] C. Use the above-mentioned time-frequency spectrum as the input of the time-frequency spectrum enhancement algorithm of the proposed fully complex convolutional neural network to obtain the enhanced complex time-frequency spectrum.

[0094] D. The enhanced complex time-frequency spectrum obtained in step C is used to form an enhanced radar action recognition data set;

[0095] E. The enhanced radar action recognition dataset obtained in step D is used as the input of the radar classification model to obtain the final specific activity type monitored by the radar.

[0096] like Figure 2The functional module schematic diagram of the system of the present invention is shown as follows: the system disclosed in the present invention for implementing the complex time-frequency spectrum enhancement method comprises a signal construction module, a signal processing module, a model construction module, a model training module and a data enhancement module; the signal construction module, the signal processing module, the model construction module, the model training module and the data enhancement module are connected in series in sequence; the signal construction module is used to construct a random signal database based on the frequency modulation signal, and upload the data information to the signal processing module; the signal processing module is used to perform a discrete short-time Fourier transform on the signal in the constructed random signal database according to the received data information to obtain a complex time-frequency spectrum data set , and upload the data information to the model construction module; the model construction module is used to construct a complex time-frequency spectrum enhancement primary model based on complex convolution and complex deconvolution according to the received data information, and upload the data information to the model training module; the model training module is used to train the complex time-frequency spectrum enhancement primary model constructed in step S3 according to the received data information and the obtained complex time-frequency spectrum data set, to obtain the complex time-frequency spectrum enhancement model, and upload the data information to the data enhancement module; the data enhancement module is used to perform actual complex time-frequency spectrum enhancement according to the received data information and the obtained complex time-frequency spectrum enhancement model.

Claims

1. A complex time-frequency spectrum enhancement method, comprising the following steps: S1. Construct a random signal database based on FM signals; S2. Performing a discrete short-time Fourier transform on the signal in the random signal database constructed in step S1 to obtain a complex time-frequency spectrum data set; S3. Based on complex convolution and complex deconvolution, a complex time-frequency spectrum enhancement primary model is constructed; S4. Using the complex time-frequency spectrum data set obtained in step S2, the complex time-frequency spectrum enhancement primary model constructed in step S3 is trained to obtain a complex time-frequency spectrum enhancement model; S5. Use the complex time-frequency spectrum enhancement model obtained in step S4 to enhance the actual complex time-frequency spectrum.

2. The method for enhancing the complex time-frequency spectrum according to claim 1, characterized in that The step S1 of constructing a random signal database based on the frequency modulation signal specifically includes the following steps: Based on the cosine frequency modulation signal, the first random signal S1(k) is constructed as Where M is the number of sub-signals; A m is the amplitude value of the mth sub-signal; B m is the first frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-100,100); C m is the second frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (0.001,10); D m is the third frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-8,8); E m is the fourth frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-5,5); θ m is the initial phase of the mth sub-signal and obeys the uniform distribution of (0,2π); k is time; Based on the four-frequency modulation signal, the second random signal S2(k) is constructed as Where F m is the fifth frequency adjustment coefficient of the mth sub-signal and is uniformly distributed in the range of (-50, 50); G m is the sixth frequency adjustment coefficient of the mth sub-signal and obeys the uniform distribution of (-25, 25); Randomly generate X groups of first random signals S1(k) and Y groups of second random signals S2(k); Gaussian white noise is added to all randomly generated signals, and the signal-to-noise ratio of the signal after adding Gaussian white noise is guaranteed to be -5dB to 40dB.

3. The method for enhancing the complex time-frequency spectrum according to claim 2, characterized in that Step S2 performs a discrete short-time Fourier transform on the signal in the random signal database constructed in step S1 to obtain a complex time-frequency spectrum data set, which specifically includes the following steps: Performing a discrete short-time Fourier transform on the signal in the random signal database constructed in step S1; The discrete short-time Fourier transform is performed using the following formula: Where N is the number of discrete short-time Fourier transform points; n is time; k is the frequency point; s(m) is the input signal; w(mn) is the set window function; Finally, the data obtained after discrete short-time Fourier transform is normalized to obtain a complex time-frequency spectrum data set.

4. The method for enhancing the complex time-frequency spectrum according to claim 3, characterized in that The step S3 described in which the complex time spectrum enhancement primary model is constructed based on complex convolution and complex deconvolution specifically includes the following steps: The constructed complex time-frequency spectrogram enhancement primary model includes 10 two-dimensional complex convolutional layers with ReLu activation function and 10 two-dimensional complex deconvolutional layers with ReLu activation function; A 2D complex convolutional layer with ReLu activation function is used to extract spectral information in the spectrogram and reduce the dimensionality of the input data; A 2D complex deconvolution layer with ReLu activation function is used to recover the lost information in the spectrogram and enhance the spectrogram; The first two-dimensional complex convolution layer with ReLu activation function to the tenth two-dimensional complex convolution layer with ReLu activation function are connected in series; The output of the 10th 2D complex convolutional layer with ReLu activation function is directly connected to the input of the 1st 2D complex deconvolutional layer with ReLu activation function; The 1st to 10th two-dimensional complex deconvolution layers with ReLu activation function are connected in series in sequence; in addition, the output of the i-th two-dimensional complex convolution layer with ReLu activation function is simultaneously connected to the input of the 11-i-th two-dimensional complex deconvolution layers with ReLu activation function; the value of i is 1 to 9; the number of output channels of the 10th two-dimensional complex deconvolution layer with ReLu activation function is 1.

5. The method for enhancing the complex time-frequency spectrum according to claim 4, characterized in that The training described in step S4 specifically includes the following steps: Construct loss function; During the training process, back propagation is performed to update the constructed model until the model converges; the model converges, specifically, the value of the constructed loss function remains unchanged within a set range.

6. The method for enhancing the complex time-frequency spectrum according to claim 5, characterized in that The step S5 adopts the complex time-frequency spectrum enhancement model obtained in step S4 to perform actual complex time-frequency spectrum enhancement, which specifically includes the following steps: Acquire a complex signal of a time-spectrogram to be enhanced; Performing discrete short-time Fourier transform on the complex signal of the acquired time-frequency spectrum to be enhanced to obtain a corresponding complex time-frequency spectrum; Normalize the obtained complex time-frequency spectrum; The normalized complex time-spectrum graph is input into the obtained complex time-spectrum graph enhancement model, and the output of the complex time-spectrum graph enhancement model is used as the enhanced complex time-spectrum graph to complete the actual enhancement of the complex time-spectrum graph.

7. A system for implementing the complex time-frequency spectrum enhancement method according to any one of claims 1 to 6, characterized in that It includes a signal construction module, a signal processing module, a model construction module, a model training module and a data enhancement module; the signal construction module, the signal processing module, the model construction module, the model training module and the data enhancement module are connected in series in sequence; the signal construction module is used to construct a random signal database based on the frequency modulation signal, and upload the data information to the signal processing module; The signal processing module is used to perform discrete short-time Fourier transform on the signal in the constructed random signal database according to the received data information, obtain a complex time-frequency spectrum data set, and upload the data information to the model construction module; The model building module is used to build a complex time-frequency spectrum enhancement primary model based on complex convolution and complex deconvolution according to the received data information, and upload the data information to the model training module; The model training module is used to train the complex time-frequency spectrum enhancement primary model constructed in step S3 according to the received data information and the obtained complex time-frequency spectrum data set, obtain the complex time-frequency spectrum enhancement model, and upload the data information to the data enhancement module; The data enhancement module is used to enhance the actual complex time-frequency spectrum diagram according to the received data information and using the obtained complex time-frequency spectrum diagram enhancement model.

Images