Radar human body motion signal loss compensation method, device, equipment and medium

Abstract

The invention provides a radar human body motion signal loss compensation method, device, equipment and medium, the method comprises the following steps: a GAN-based data filling model GT-GAIN is constructed, a generator G and a discriminator D of the model each comprise a single-layer bidirectional GRU, a Transformer encoder and a linear normalization layer LayerNorm, and the discriminator D further comprises a linear layer Linear; obtaining a missing training set Xtrain, and training a model GT-GAIN after random numerical value filling is carried out on the missing part of the training set Xtrain; and for missing electromagnetic interference data s, performing noise removal and filling by using the trained model GT-GAIN, and realizing data recovery and completion to obtain output data S. According to the method, random numerical values are used for filling missing parts, the missing parts are regarded as noise data to be removed and filled, and the generated radar signals are used for replacing lost data signals, so that the effect of signal compensation is achieved.

Description

Technical Field

[0001] This invention belongs to the field of signal processing technology, and in particular relates to a method, device, equipment and medium for compensating for the loss of radar human motion signals. Background Technology

[0002] Autonomous driving systems are typical artificial intelligence systems. Millimeter-wave radar sensors, as representative sensors in autonomous driving systems, mainly play a role in assisting driving. When they are subjected to electromagnetic interference, it may cause the autonomous driving system to make misjudgments, or even cause major traffic safety accidents.

[0003] In communication systems, when strong interference signals are simultaneously connected to a useful signal, the receiver gain decreases, causing blocking interference and preventing the reception of the useful signal, resulting in signal loss. Similarly, in electromagnetic interference environments, radar continuous waves can generate signals that cannot change abruptly, i.e., there is a data noise problem.

[0004] Traditionally, missing value handling methods can be categorized into two types. The first method attempts to ignore partially missing observations, but this can lead to significant bias and inefficiency. The other approach addresses missing data by making reasonable inferences from observations to explain the missing data. Common missing value handling methods include those based on traditional statistics, such as multiple imputation, hot-card imputation, and regression imputation; machine learning-based methods include K-nearest neighbor imputation and RF prediction of missing values; and deep learning-based imputation includes single-layer neural networks, stacked bidirectional and unidirectional LSTM recurrent neural networks, and adaptive neural networks (ADNN). However, current technologies lack measures to address temporal relationships and cannot effectively learn the dynamic spatiotemporal dependencies in time-series data, thus hindering accurate data imputation. Summary of the Invention

[0005] The purpose of this invention is to provide a method, device, equipment, and medium for compensating for the loss of human motion signals in millimeter-wave radar based on GAN, in order to remove noise from electromagnetic interference data for human motion recognition, in order to address the shortcomings of existing technologies.

[0006] The above objectives are achieved through the following technical solutions:

[0007] A method for compensating for lost radar human motion signals includes the following steps:

[0008] Step S1: Construct a GAN-based data imputation model GT-GAIN. The generator G and discriminator D of this model both include a single-layer bidirectional GRU, a Transformer encoder, and a linear normalization layer LayerNorm. The discriminator D also includes a linear layer Linear.

[0009] Step S2: Obtain the training set X with missing parts train After randomly imputing the missing parts, the training data is used to impute the GT-GAIN model, which includes the following steps:

[0010] S2-1: Generate a matrix M with a uniform distribution region of [0,1). mid And based on the missing rate P miss matrix M mid Process the missing matrix into a random discrete matrix M; then compare the missing matrix M with the training set X. train Perform matrix multiplication to obtain the dataset X containing the missing parts. M ;

[0011] S2-2: Generate a matrix Z with a uniform distribution region of [0, 0.01). mid , matrix Z mid Perform matrix multiplication with 1-M to obtain random imputation value Z, and then use the dataset X. M Adding the random fill value Z to obtain the random fill matrix X;

[0012] S2-3: Add the missing matrix M to the randomly filled matrix X to obtain matrix XM, and use matrix XM as input data to train the model;

[0013] Step S3: For missing electromagnetic interference data s, the input data is processed in a similar manner to steps S2-1 to S2-3 above. Then, the trained data imputation model GT-GAIN is used to remove and impute noise, thereby restoring and completing the data to obtain the output data S.

[0014] Preferably, in step S1, the data processing procedure of the data imputation model GT-GAIN is as follows: Input data is input into a single-layer Bi-GRU of generator G for feature extraction, and the extracted data feature A is sent into the Transformer encoder of generator G to further extract data feature B; data feature A and data feature B are matrix superimposed, and the resulting superimposed matrix is ​​sent into the linear normalization layer LayerNorm of generator G for dimensionality compression to obtain data feature C; data feature C is used as the input of discriminator D and similar processing is performed to obtain data feature D, and data feature D is then processed by the linear layer Linear of discriminator D to obtain the output result Output.

[0015] Preferably, in step S2-1, the training set X train The generation method is as follows: For a given complete dataset Data, randomly select a portion of the data as the original data Y. train Then, for the original data Y train Normalize the data to obtain the training set Xtrain .

[0016] Preferably, in step S2-1, the method for generating the missing matrix M is as follows: for matrix M mid If matrix M mid The value in is greater than P miss If the matrix M is 1, then set it to 1, indicating that the data exists. mid The value in is less than P miss If we set it to 0, it means data is missing, resulting in a random discrete missing matrix M.

[0017] Preferably, in the data imputation model GT-GAIN,

[0018] The loss function G of the generator G loss for:

[0019] G loss = (1-α)×D loss +α×G loss-mid

[0020] In the above formula, D loss G represents the loss function result of the discriminator D. loss-mid The loss function result of generator G is given, and α is a hyperparameter that balances the two, with a value between 0 and 1.

[0021] G loss-mid =MSE(G(XM),Y)+||(1-M)⊙(G(XM)-X)||2

[0022] In the above formula, Y represents the original data, MSE represents the mean squared error loss function, G(XM) represents the generator result, used to calculate the loss value between the output value of generator G and the original data; ||·||2 represents the L2 norm, used to calculate the distance between the generated results in the values ​​generated by generator G.

[0023] The loss function D of discriminator D loss for:

[0024] D loss =D(Y) loss +D(G(XM)) loss

[0025] In the above formula,

[0026] D(G(XM)) loss =BCE(D(G(XM)), M)

[0027] D(Y) loss =BCE(D(Y), One)

[0028] In the above two equations, BCE is the binary cross-entropy loss function, D(G(XM)) is the discrimination result of the discriminator D on the generator result G(XM); D(Y) represents the discrimination result of the discriminator D on the original data Y, and One is a matrix of all 1s.

[0029] A radar human motion signal loss compensation device includes:

[0030] The model building module is used to build the GAN-based data imputation model GT-GAIN. The generator G and discriminator D of this model both include a single-layer bidirectional GRU, a Transformer encoder and a linear normalization layer LayerNorm. The discriminator D also includes a linear layer Linear.

[0031] The model training module is used to obtain the training set X with missing parts. train After randomly filling in the missing parts, the training data is used to fill in the GT-GAIN model to obtain the trained model.

[0032] The data generation module is used to randomly fill in the missing electromagnetic interference data s with random values, and then use the trained data imputation model GT-GAIN to remove noise and fill in the missing parts, so as to restore and complete the data to obtain the output data S.

[0033] An electronic device, comprising:

[0034] Memory, used to store computer programs;

[0035] A processor is configured to execute the computer program to implement the steps of the radar human motion signal loss compensation method disclosed above.

[0036] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the radar human motion signal loss compensation method disclosed above.

[0037] The beneficial effects of this invention are:

[0038] This invention proposes a method for compensating for lost human motion signals in millimeter-wave radar, combining the concepts of generative adversarial networks (GANs), Transformer encoders, and Bi-GRUs. By constructing a GT-GAIN model to learn the overall distribution of a multivariate time-series dataset, the generator is responsible for generating relevant values ​​based on the input, while the discriminator is responsible for element-wise identification of whether the input is a generated value. A gated recurrent neural network is used to process incomplete time series data, obtaining higher-level data features. The Transformer encoder is used to obtain more intermediate dependencies, enhancing both generation and discrimination capabilities. This method fully learns the distribution patterns of time-series data by capturing the dependencies between future and past information for each missing radar time-series data point, as well as long-term sequence data. Random values ​​are used to fill in the missing parts, treating them as noise data for removal and imputation. The generated radar signal replaces the lost data signal, achieving signal compensation and thus fulfilling the task of compensating for lost data due to electromagnetic interference. Attached Figure Description

[0039] Figure 1 This is a structural diagram of the generator and discriminator of the data imputation model GT-GAIN proposed in this invention;

[0040] Figure 2 This is a structural diagram of the data imputation model GT-GAIN proposed in this invention;

[0041] Figure 3 This describes the data preprocessing process for the GT-GAIN data imputation model proposed in this invention. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0043] In this invention, the serial numbers assigned to components, such as "first" and "second," are merely used to distinguish the described objects and do not have any sequential or technical meaning. The terms "connection" and "linkage" used in this invention, unless otherwise specified, include both direct and indirect connections (linkages). It should be understood that the terms "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention.

[0044] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0045] This invention provides a method for compensating for the loss of radar human motion signals, comprising the following steps:

[0046] Step S1: Construct a GAN-based data imputation model GT-GAIN. The generator G and discriminator D of this model both include a single-layer bidirectional GRU, a Transformer encoder, and a linear normalization layer LayerNorm. The discriminator D also includes a linear layer, such as... Figure 1 As shown.

[0047] The data processing procedure of the data imputation model GT-GAIN is as follows: Input data is fed into a single-layer Bi-GRU of generator G for feature extraction, and the extracted data feature A is fed into the Transformer encoder of generator G to further extract data feature B; data feature A and data feature B are matrix superimposed, and the resulting superimposed matrix is ​​fed into the linear normalization layer LayerNorm of generator G for dimensionality compression to obtain data feature C; data feature C is used as the input of discriminator D and similar processing is performed to obtain data feature D, and data feature D is then processed by the linear layer Linear of discriminator D to obtain the output result Output.

[0048] The proposed data imputation model GT-GAIN mainly draws on the structure of GAN models and the data processing method of GAIN. Its generator G and discriminator D are composed of a gated recurrent neural network (GRU) and an encoder of an autonomous attention network, which can efficiently and accurately impute missing data directly on incomplete datasets. GT-GAIN utilizes the attention mechanism and Bi-GRU network to learn long-term dependency features of data. Features containing important information do not disappear as the step size increases, thus highlighting the influence of more important information and making the framework easier to learn long-term sequences. The interdependence of distances solves the problem of information loss caused by excessively long data. Finally, the missing data is imputed through iterative updates of the generator G and discriminator D.

[0049] Step S2: Obtain the training set X with missing partstrain After randomly filling in the missing parts, the training data filler model GT-GAIN is trained.

[0050] like Figure 3 As shown, the data preprocessing process for this GT-GAIN model is as follows:

[0051] (1) Training set X train Generation: For the complete dataset Data, randomly select a portion of the data as the original data Y. train , used to generate matrix X of missing data M Then, for the original data Y... train Data standardization (normalization) is performed to normalize the data values ​​to a range of 0 to 1, resulting in the training set X. train .

[0052] (2) Generation of the missing matrix M: based on the missing rate P miss Generate a random discrete missing matrix M. First, generate a matrix M with a uniform distribution region of [0,1). mid Then, if matrix M mid The value in is greater than P miss If the matrix M is 1, then set it to 1, indicating that the data exists. mid The value in is less than P miss If the number of zeros is zero, it is set to 0, indicating data loss, and a random discrete missing matrix M is generated. In the missing matrix M, the ratio of the number of zeros to the total number of elements is P. miss .

[0053] (3) Dataset X containing missing parts M Generation: Combine the missing matrix M and the training set X train Perform matrix multiplication to obtain the dataset X containing the missing parts. M .

[0054] (4) Generation of randomly imputed matrix X: Since deep learning models cannot handle null values ​​(NAN), and calculation errors are prone to occur when there are a large number of 0 values ​​in the data, it is necessary to first impute the missing data randomly. First, generate a matrix Z with a uniform distribution region of [0, 0.01). mid Multiply it by 1-M to obtain the random imputation value Z, and then use the dataset X M Adding the random fill value Z to obtain the random fill matrix X.

[0055] like Figure 2As shown, in GT-GAIN, the randomly imputed data (i.e., the randomly imputed matrix X) and the missing matrix M are superimposed to obtain matrix XM. Using matrix XM as input to the generator G enhances the generator G's ability to distinguish between the original data and randomly generated data in the randomly imputed matrix X, allowing the generator G to focus on different aspects. The discriminator D simultaneously distinguishes between the generator G's result G(XM) and the original data Y, improving the generator G's generation ability and minimizing the influence of the original data on the result. Here, the matrix One, consisting entirely of 1s, represents all data in the original data Y as original data.

[0056] The loss function G of the generator G loss for:

[0057] G loss = (1-α)×D loss +α×G loss-mid

[0058] In the above formula, D loss G represents the loss function result of the discriminator D. loss-mid Let G be the loss function result of the generator G, and α be a hyperparameter balancing the two, with a value between 0 and 1. Since the parameter update of the generator G is affected by both its own result and the discriminator result, it is necessary to convert the loss function result G of the generator G into a loss function result G. loss-mid The loss function result D of the discriminator D loss Weighted summation.

[0059] G loss-mid =MSE(G(XM),Y)+||(1-M)⊙(G(XM)-X)||2

[0060] In the above formula, Y represents the original data, MSE represents the mean squared error loss function, G(XM) represents the generator result, used to calculate the loss value between the output value of the generator G and the original data; ||·||2 represents the L2 norm, used to calculate the distance between the generated results in the generated values;

[0061] The loss function D of discriminator D loss for:

[0062] D loss =D(Y) loss +D(G(XMM)) loss

[0063] In the above formula,

[0064] D(G(XM)) loss =BCE(D(G(XM)), M)

[0065] D(Y) loss=BCE(D(Y), One)

[0066] BCE is the binary cross-entropy loss function, D(G(XM)) is the discrimination result of discriminator D on generator result G(XM); D(Y) represents the discrimination result of discriminator D on original data Y, and One is a matrix of all 1s.

[0067] Since the goal of discriminator D is to distinguish whether the input value is a generated value (resulting in "0") or an original value (resulting in "1"), it can be compared to a multi-class classification problem. This is why discriminator D uses the sigmoid activation function. To reduce the influence of the original data on the generator G, the loss function of discriminator D also consists of the sum of two parts.

[0068] To reduce the impact of the original data Y on the generator D and the discriminator G, the loss function of the generator D includes the calculation of the non-missing value part; the discriminator G discriminates the generator D and the original data Y at the same time.

[0069] Step S3: For missing electromagnetic interference data s, the input data is processed in a similar manner to steps S2-1 to S2-3 above. Then, the trained data imputation model GT-GAIN is used to remove and impute noise, thereby restoring and completing the data to obtain the output data S.

[0070] The present invention also provides a radar human motion signal loss compensation device, comprising:

[0071] The model building module is used to build the GAN-based data imputation model GT-GAIN. The generator G and discriminator D of this model both include a single-layer bidirectional GRU, a Transformer encoder and a linear normalization layer LayerNorm. The discriminator D also includes a linear layer Linear.

[0072] The model training module is used to obtain the training set X with missing parts. train After randomly filling in the missing parts, the training data is used to fill in the GT-GAIN model to obtain the trained model.

[0073] The data generation module is used to randomly fill in the missing electromagnetic interference data s with random values, and then use the trained data imputation model GT-GAIN to remove noise and fill in the missing parts, so as to restore and complete the data to obtain the output data S.

[0074] The present invention also provides an electronic device, comprising:

[0075] Memory, used to store computer programs;

[0076] A processor is configured to execute the computer program to implement the steps of the radar human motion signal loss compensation method as previously disclosed.

[0077] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the radar human motion signal loss compensation method disclosed above.

[0078] The pseudocode for the GT-GAIN model is shown in the table below:

[0079]

[0080] For electromagnetic signal data with varying missing rates, the GT-GAIN model of this invention can effectively improve the classification results of electromagnetic signal images after randomly imputing missing data, and the repaired results are relatively stable. Furthermore, imputing the missing data with data generated by the generator can effectively reduce interference and improve the accuracy of the repair.

[0081] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for compensating for the loss of radar human motion signals, characterized in that, Includes the following steps: Step S1: Construct a GAN-based data imputation model GT-GAIN. The generator G and discriminator D of this model both include a single-layer bidirectional GRU, a Transformer encoder, and a linear normalization layer LayerNorm. The discriminator D also includes a linear layer Linear. Step S2: Obtain the training set X with missing parts train After randomly imputing the missing parts, the training data is used to impute the GT-GAIN model, which includes the following steps: S2-1: Generate a matrix M with a uniform distribution region of [0,1). mid And based on the missing rate P miss matrix M mid Process it into a random discrete missing matrix M; Compare the missing matrix M with the training set X train Perform matrix multiplication to obtain the dataset X containing the missing parts. M ; S2-2: Generate a matrix Z with a uniform distribution region of [0, 0.01). mid , matrix Z mid Perform matrix multiplication with 1-M to obtain random imputation value Z, and then use the dataset X. M Adding the random fill value Z to obtain the random fill matrix X; S2-3: Add the missing matrix M to the randomly filled matrix X to obtain matrix XM, and use matrix XM as input data to train the model; Step S3: For missing electromagnetic interference data s, the input data is processed in a similar manner to steps S2-1 to S2-3 above. Then, the trained data imputation model GT-GAIN is used to remove and imput noise, thereby restoring and completing the data to obtain the output data S.

2. The radar human motion signal loss compensation method according to claim 1, characterized in that, In step S1, the data processing procedure of the data imputation model GT-GAIN is as follows: Input data is input into a single-layer Bi-GRU of generator G for feature extraction, and the extracted data feature A is sent into the Transformer encoder of generator G to further extract data feature B; data feature A and data feature B are matrix superimposed, and the resulting superimposed matrix is ​​sent into the linear normalization layer LayerNorm of generator G for dimensionality compression to obtain data feature C; data feature C is used as the input of discriminator D and similar processing is performed to obtain data feature D, and data feature D is then processed by the linear layer Linear of discriminator D to obtain the output result Output.

3. The radar human motion signal loss compensation method according to claim 1, characterized in that, In step S2-1, the training set X train The generation method is as follows: For a given complete dataset Data, randomly select a portion of the data as the original data Y. train Then, for the original data Y train Normalize the data to obtain the training set X train .

4. The radar human motion signal loss compensation method according to claim 1, characterized in that, In step S2-1, the method for generating the missing matrix M is as follows: For matrix M mid If matrix M mid The value in is greater than P miss If the matrix M is 1, then set it to 1, indicating that the data exists. mid The value in is less than P miss If we set it to 0, it means data is missing, resulting in a random discrete missing matrix M.

5. The radar human motion signal loss compensation method according to claim 1, characterized in that, In the data imputation model GT-GAIN The loss function G of the generator G loss for: G loss =(1-a)×D loss +α×G loss-mid In the above formula, D loss G represents the loss function result of the discriminator D. loss-mid The loss function result of generator G is given, and α is a hyperparameter that balances the two, with a value between 0 and 1. G loss-mid =MSE(G(XM),Y)+||(1-M)⊙(G(XM)-X)||2 In the above formula, Y represents the original data, MSE represents the mean squared error loss function, G(XM) represents the generator result, used to calculate the loss value between the output value of generator G and the original data; ||·||2 represents the L2 norm, used to calculate the distance between the generated results in the values ​​generated by generator G. The loss function D of discriminator D loss for: D loss =D(Y) loss +D(G(XM)) loss In the above formula, D(D(XM)) loss =BCE(D(G(XM)),M) D(Y) loss =BCE(D(Y),One) In the above two equations, BCE is the binary cross-entropy loss function, D(G(XM)) is the discrimination result of the discriminator D on the generator result G(XM); D(Y) represents the discrimination result of the discriminator D on the original data Y, and One is a matrix of all 1s.

6. A radar human motion signal loss compensation device, characterized in that, include: The model building module is used to build the GAN-based data imputation model GT-GAIN. The generator G and discriminator D of this model both include a single-layer bidirectional GRU, a Transformer encoder and a linear normalization layer LayerNorm. The discriminator D also includes a linear layer Linear. The model training module is used to obtain the training set X with missing parts. train After randomly filling in the missing parts, the training data is used to fill in the GT-GAIN model to obtain the trained model. The data generation module is used to randomly fill in the missing electromagnetic interference data s with random values, and then use the trained data imputation model GT-GAIN to remove noise and fill in the missing parts, so as to restore and complete the data to obtain the output data S.

7. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the radar human motion signal loss compensation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the radar human motion signal loss compensation method as described in any one of claims 1 to 5.

Images