Keyboard electromagnetic leakage signal noise reduction method and system based on generative adversarial network

Abstract

The present invention discloses a keyboard electromagnetic leakage signal denoising method and system based on a generative adversarial network, which belongs to the field of signal denoising. The method comprises the following steps: S1, collecting the original noisy leakage signal of the keyboard and generating data to be denoised; the original noisy leakage signal includes: the original electromagnetic signal and the environmental noise signal; S2, constructing a SADAN network model, denoising the data to be denoised based on the SADAN network model, and completing the keyboard electromagnetic leakage signal denoising process. The SADAN network model of the present invention includes a discriminator and a denoiser. The discriminator can distinguish the authenticity of the original electromagnetic signal of the keyboard; the denoiser can suppress the noise in the noisy signal of the keyboard, output the denoised signal, and the discriminator judges the denoising effect; the introduction of the self-attention mechanism enables the model to have a global analysis capability for the keyboard leakage signal sequence; a new loss system is constructed to make network optimization more efficient.

Description

Technical Field

[0001] The present invention belongs to the technical field of signal noise reduction, and in particular relates to a method and system for reducing the noise of keyboard electromagnetic leakage signals based on a generative adversarial network. Background Art

[0002] When processing information, electronic devices such as computers and communication terminals inevitably emit electromagnetic waves into their surroundings, generating unintentional, non-communicative electromagnetic radiation. These unintentionally emitted electromagnetic signals have rich spectral components and carry a wealth of useful information, posing a potential risk of information leakage.

[0003] The keyboard is a commonly used computer input device used to input data information into the host terminal. This data information, such as characters, numbers, commands, etc., is very likely to contain important content, such as sensitive information such as account passwords. There is a risk of electromagnetic information leakage during the transmission process from the keyboard to the host, posing a great threat to information security. Existing keyboard electromagnetic leakage detection and identification methods usually require the analysis of electromagnetic signals under the premise of known electromagnetic leakage information characteristics and specific environmental conditions. They have weak noise resistance and insufficient generalization ability. Various aspects affect the effectiveness of detection and identification work:

[0004] (1) The detection and recognition of computer keyboard key information is highly dependent on human experience, and therefore lacks a means of automatic recognition and positioning;

[0005] (2) In a complex electromagnetic environment, the electromagnetic leakage signal has a low signal-to-noise ratio and is easily interfered by environmental noise. When faced with a complex electromagnetic environment, the intercepted electromagnetic information contains a large amount of noise components, which hinders the detection process;

[0006] (3) Most of the detection work related to electromagnetic leakage takes risk assessment as the main purpose, and the technical goals are mainly focused on the question of whether there is a leakage risk, that is, the question of "whether there is a leakage risk". There is a lack of research on further detection and identification of leakage signals, that is, the question of "what is it" and where the leakage information is located in the signal.

[0007] Therefore, the detection of electromagnetic leakage information of computer keyboards is to confirm the existence of this risk, and in-depth research on it has important scientific research value and practical significance. Summary of the Invention

[0008] To address the problem of low signal-to-noise ratio and difficulty in detection and identification of keyboard leakage signals in complex electromagnetic environments, this paper proposes an algorithm suitable for denoising keyboard leakage signals - SADAN. Based on the idea of ​​adversarial game, this algorithm utilizes generative adversarial networks and self-attention mechanism to continuously enhance the discrimination and denoising capabilities of the discriminator and denoiser during the learning process.

[0009] To achieve the above objectives, the present invention provides the following solution: a keyboard electromagnetic leakage signal noise reduction method based on a generative adversarial network, comprising the following steps:

[0010] S1. Collecting the original noisy leakage signal of the keyboard to generate data to be denoised; the original noisy leakage signal includes: an original electromagnetic signal and an environmental noise signal;

[0011] S2. Construct a SADAN network model, and perform denoising on the data to be denoised based on the SADAN network model to complete the noise reduction processing of the keyboard electromagnetic leakage signal.

[0012] Preferably, the SADAN network model includes: a denoiser and a discriminator;

[0013] The denoiser is composed of a convolution layer, a self-attention layer, a deconvolution layer and a ReLU activation layer; the denoiser is used to denoise the data to be denoised to obtain a denoised signal;

[0014] The discriminator is composed of a convolutional layer, a self-attention layer and a LeakyReLU activation layer; the discriminator is used to judge the authenticity of the denoised signal and output the denoised signal that is judged to be true.

[0015] Preferably, the denoiser uses mean square error as the loss function:

[0016]

[0017] Where, L G represents the denoiser loss function, m represents the logarithm of the combination of the ambient noise slice and the keyboard original electromagnetic signal slice, x i Represents the i-th signal value in the original electromagnetic signal of the keyboard, y i represents the i-th signal value in the keyboard noisy leakage signal, and G represents the mapping established by the denoiser from the keyboard noisy leakage signal to the denoised signal.

[0018] Preferably, the discriminator adopts a hinge loss function:

[0019] L D =-Ε x～pt [min(0,-1+D(x))]-Ε g～pt [min(0,-1-D(g))]

[0020] Where, L D represents the discriminator loss function; E represents the mathematical expectation; P tRepresents the distribution of the original electromagnetic signal of the keyboard, x represents the original electromagnetic signal of the keyboard, g represents the denoised signal, and D is the mapping established by the discriminator to discriminate the input x and g.

[0021] Preferably, the training method of the SADAN network model includes:

[0022] S21, inputting the original electromagnetic signal into the discriminator for training and learning;

[0023] S22, inputting the slices of the original noisy leakage signal and the original electromagnetic signal into a denoiser, outputting the denoised signal, and inputting the denoised signal into the discriminator to determine authenticity;

[0024] S23, calculating the loss function value of the SADAN network model, and back-propagating the loss error to optimize and update the parameters of the SADAN network model;

[0025] S24. Repeat S21-S23. When the loss error is stable, the SADAN network model training is completed.

[0026] The present invention also provides a keyboard electromagnetic leakage signal noise reduction system based on a generative adversarial network, comprising: an acquisition module and a denoising module;

[0027] The acquisition module is used to collect the original noisy leakage signal of the keyboard and generate the data to be denoised; the original noisy leakage signal includes: the original electromagnetic signal and the environmental noise signal;

[0028] The denoising module is used to construct a SADAN network model and denoise the data to be denoised based on the SADAN network model to complete the noise reduction processing of the keyboard electromagnetic leakage signal.

[0029] Preferably, the SADAN network model includes: a denoiser and a discriminator;

[0030] The denoiser is composed of a convolution layer, a self-attention layer, a deconvolution layer and a ReLU activation layer; the denoiser is used to denoise the data to be denoised to obtain a denoised signal;

[0031] The discriminator is composed of a convolutional layer, a self-attention layer and a LeakyReLU activation layer; the discriminator is used to judge the authenticity of the denoised signal and output the denoised signal that is judged to be true.

[0032] Preferably, the denoiser uses mean square error as the loss function:

[0033]

[0034] Where, L G represents the denoiser loss function, m represents the logarithm of the combination of the ambient noise slice and the keyboard original electromagnetic signal slice, x i Represents the i-th signal value in the original electromagnetic signal of the keyboard, y i represents the i-th signal value in the keyboard noisy leakage signal, and G represents the mapping established by the denoiser from the keyboard noisy leakage signal to the denoised signal.

[0035] Preferably, the discriminator adopts a hinge loss function:

[0036] L D =-Ε x～pt [min(0,-1+D(x))]-Ε g～pt [min(0,-1-D(g))]

[0037] Where, L D represents the discriminator loss function; E represents the mathematical expectation; P t Represents the distribution of the original electromagnetic signal of the keyboard, x represents the original electromagnetic signal of the keyboard, g represents the denoised signal, and D is the mapping established by the discriminator to discriminate the input x and g.

[0038] Preferably, the denoising module trains the SADAN network model in a method comprising:

[0039] Step 1: inputting the original electromagnetic signal into the discriminator for training and learning;

[0040] Step 2: Inputting the slices of the original noisy leakage signal and the original electromagnetic signal into a denoiser, outputting the denoised signal, and inputting the denoised signal into the discriminator to determine authenticity;

[0041] Step 3: Calculate the loss function value of the SADAN network model, and back-propagate the loss error to optimize and update the parameters of the SADAN network model;

[0042] Step 4: Repeat steps 1 to 3. When the loss error is stable, the SADAN network model training is completed.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The proposed algorithm, SADAN, for denoising keyboard radiation-coupled leakage signals, consists of a discriminator and a denoiser. The discriminator identifies the authenticity of the keyboard's original electromagnetic signal; the denoiser suppresses noise from the noisy keyboard signal, outputting a denoised signal that the discriminator then determines the effectiveness of the denoising. The introduction of a self-attention mechanism enables the model to globally analyze keyboard leakage signal sequences, while a novel loss system is constructed to enhance network optimization efficiency. Finally, the denoised signal was input into the GRU-CN detection algorithm for detection and recognition, achieving excellent results, demonstrating the effectiveness of SADAN as a keyboard electromagnetic leakage signal denoising method. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 This is a flow chart of a keyboard electromagnetic leakage signal noise reduction method based on a generative adversarial network according to an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram of the SADAN network model structure according to an embodiment of the present invention;

[0048] Figure 3 Schematic diagram of the Loss value reduction process in the SADAN network model training according to an embodiment of the present invention;

[0049] Figure 4 Schematic diagram showing the waveform comparison of the denoised signal obtained by using the SADAN network model for the keyboard key H according to an embodiment of the present invention, the original electromagnetic signal of the keyboard, and the simulated noisy signal;

[0050] Figure 5 Schematic diagram comparing the leakage signal and the original electromagnetic signal waveform of button H according to an embodiment of the present invention;

[0051] Figure 6 Schematic diagram showing the comparison between the denoised signal output by the SADAN network model denoiser and the original signal and the leakage signal according to an embodiment of the present invention;

[0052] Figure 7 Schematic diagram of a text recognition test sample according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0054] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Example 1:

[0056] like Figure 1 As shown, this embodiment provides a keyboard electromagnetic leakage signal noise reduction method based on a generative adversarial network, comprising the following steps:

[0057] S1. Collect the original noisy leakage signal of the keyboard to generate data to be noise-reduced; the original noisy leakage signal includes: an original electromagnetic signal and an environmental noise signal.

[0058] In this embodiment, the keyboard connection line is coupled with a caliper when no keys are pressed to obtain the ambient noise signal during keyboard operation, and then superimposed with the original electromagnetic signal of the keyboard to obtain the original noisy leakage signal of the keyboard, as shown in the following formula:

[0059] y=n+x

[0060] Where y represents the keyboard noise leakage signal, n represents the ambient noise signal, and x represents the keyboard original electromagnetic signal.

[0061] S2. Construct a SADAN network model, denoise the data to be denoised based on the SADAN network model, and complete the noise reduction processing of the keyboard electromagnetic leakage signal.

[0062] like Figure 2 The figure shows the structure of the SADAN network model of this embodiment. Specifically, the SADAN network model includes a denoiser and a discriminator. The denoiser is composed of a convolutional layer, a self-attention layer, a deconvolution layer, and a ReLU activation layer. It is used to denoise the data to be denoised to obtain a denoised signal. From the principle analysis, it can be seen that the denoiser is a mapping from the keyboard noisy leakage signal to the denoised signal, which can be expressed as:

[0063] g=G(y;θ)

[0064] Where y represents the noisy keyboard leakage signal, θ represents the denoiser network parameters, g represents the denoised signal, and G is the mapping established by the denoiser. As can be seen from the above equation, is the final denoiser optimization direction as g becomes increasingly close to the keyboard's original electromagnetic signal x.

[0065] The discriminator consists of a convolutional layer, a self-attention layer, and a LeakyReLU activation layer; it is used to determine the authenticity of the denoised signal and output the denoised signal judged to be true. The SADAN network parameters are: kernel size of 4, filter size of 64, the sum of the number of convolutional and deconvolutional layers as the denoiser's network depth, which is 10, and the discriminator's network depth, i.e., the number of convolutional layers, is 8. The learning rate range is [10-3, 10-5], the epoch is set to 20, and the batch size is 128. The basic function formula of the LeakyReLU activation layer is as follows:

[0066]

[0067] Where, is a small constant; in this embodiment, it is set to 0.01. The purpose is to correct the zero-gradient problem of negative values ​​by treating very small linear components in x as negative inputs. The discriminator inputs the original keyboard electromagnetic signal, allowing the convolutional layer to continuously learn its probability distribution, thereby determining whether the probability distribution of the input denoised signal matches that of the original keyboard electromagnetic signal. If so, the denoised signal is judged as true, confirming the denoising ability of the denoiser. Otherwise, it is judged as false, negating the denoising effect of the denoiser.

[0068] Among them, the denoiser uses mean square error as the loss function:

[0069]

[0070] Where, L G represents the denoiser loss function, m represents the logarithm of the combination of the ambient noise slice and the keyboard original electromagnetic signal slice, x i Represents the i-th signal value in the original electromagnetic signal of the keyboard, y i represents the i-th signal value in the keyboard noisy leakage signal, and G represents the mapping established by the denoiser from the keyboard noisy leakage signal to the denoised signal.

[0071] The discriminator adopts the hinge loss function:

[0072]

[0073] Where, L D represents the discriminator loss function; E represents the mathematical expectation; P trepresents the distribution of the keyboard's raw electromagnetic signal, x represents the keyboard's raw electromagnetic signal, g represents the denoised signal, and D is the mapping established by the discriminator, which discriminates between inputs x and g. When the discriminator outputs an output greater than or equal to 1 or less than or equal to -1 for input x or g, it indicates that the discriminator's judgment of x or g is definitely true or false, and the loss function is 0. If the output is between greater than -1 and less than 1, the discriminator's judgment is uncertain, and the loss function is non-zero. Therefore, it is necessary to accumulate the losses to optimize the discriminator parameters and enable the discriminator to more accurately distinguish samples.

[0074] It can be seen from the loss functions of the denoiser and the discriminator that when the probability distributions of the denoised signal and the original electromagnetic signal of the keyboard are infinitely close, the loss function of the denoiser approaches 0; when the probability distribution of the discriminator input x is infinitely close to the distribution of the original electromagnetic signal of the keyboard, and the input g is infinitely far from the distribution of the original electromagnetic signal of the keyboard, the loss function of the discriminator approaches 0.

[0075] Then the loss function of SADAN is:

[0076] L SADAN =L G +L D

[0077] In order to obtain the best denoising effect, the parameter optimization direction of the denoiser is to minimize L G , the parameter optimization direction of the discriminator is to minimize L D .

[0078] In this embodiment, the training method of the SADAN network model includes:

[0079] S21, inputting the original electromagnetic signal into the discriminator for training and learning its probability distribution;

[0080] S22, inputting the slices of the original noisy leakage signal and the original electromagnetic signal into a denoiser, outputting a denoised signal, and inputting the denoised signal into a discriminator to determine authenticity;

[0081] Specifically, a random slice of the keyboard's original electromagnetic signal and a random slice of the ambient noise signal are selected, respectively. These two slices are combined to form the original noisy leakage signal. These two slices are then fed into the denoiser as input data and corresponding label data, respectively. After mapping by the denoiser, the denoised signal is output and fed into the discriminator for authenticity determination.

[0082] S23. Calculate the loss function value of the SADAN network model, use the gradient descent algorithm to backpropagate the loss error, and update the parameters of the SADAN network model through Adam optimization;

[0083] S24. Repeat S21-S23. When the loss error stabilizes, the SADAN network model training is complete. At this point, the denoiser in the SADAN network model has acquired the ability to "forge" denoised signals, and the discriminator has acquired the ability to distinguish between genuine and fake signals.

[0084] The trained SADAN network model is used to suppress the environmental noise in the keyboard noisy leakage signal, thereby enhancing the effective signal part in the original electromagnetic signal of the keyboard and achieving the purpose of denoising.

[0085] Example 2

[0086] This embodiment provides a keyboard electromagnetic leakage signal noise reduction system based on a generative adversarial network, including: an acquisition module and a denoising module.

[0087] The acquisition module is used to collect the original noisy leakage signal of the keyboard and generate the data to be noise-reduced; the original noisy leakage signal includes: the original electromagnetic signal and the environmental noise signal.

[0088] In this embodiment, the keyboard connection line is coupled with a caliper when no keys are pressed to obtain the ambient noise signal during keyboard operation, and then superimposed with the original electromagnetic signal of the keyboard to obtain the original noisy leakage signal of the keyboard, as shown in the following formula:

[0089] y=n+x

[0090] Where y represents the keyboard noise leakage signal, n represents the ambient noise signal, and x represents the keyboard original electromagnetic signal.

[0091] The denoising module is used to build a SADAN network model and denoise the noise reduction data based on the SADAN network model to complete the noise reduction processing of the keyboard electromagnetic leakage signal.

[0092] Specifically, the SADAN network model includes a denoiser and a discriminator. The denoiser consists of a convolutional layer, a self-attention layer, a deconvolution layer, and a ReLU activation layer. It is used to denoise the data to be denoised and obtain a denoised signal. From the principle analysis, it can be seen that the denoiser is a mapping from the keyboard noisy leakage signal to the denoised signal, which can be expressed as:

[0093] g=G(x;θ)

[0094] Where θ represents the denoiser network parameters, g is the denoised signal, and G is the mapping created by the denoiser. As can be seen from the above equation, θ is the final denoiser optimization direction as g becomes increasingly close to the keyboard's original electromagnetic signal x.

[0095] The discriminator consists of a convolutional layer, a self-attention layer, and a LeakyReLU activation layer; it is used to determine the authenticity of the denoised signal and output the denoised signal judged to be true. The SADAN network parameters are: kernel size of 4, filter size of 64, the sum of the number of convolutional and deconvolutional layers as the denoiser's network depth, which is 10, and the discriminator's network depth, i.e., the number of convolutional layers, is 8. The learning rate range is [10-3, 10-5], the epoch is set to 20, and the batch size is 128. The basic function formula of the LeakyReLU activation layer is as follows:

[0096]

[0097] Where, is a small constant; in this embodiment, it is set to 0.01. The purpose is to correct the zero-gradient problem of negative values ​​by treating very small linear components in x as negative inputs. The discriminator inputs the original keyboard electromagnetic signal, allowing the convolutional layer to continuously learn its probability distribution, thereby determining whether the probability distribution of the input denoised signal matches that of the original keyboard electromagnetic signal. If so, the denoised signal is judged as true, confirming the denoising ability of the denoiser. Otherwise, it is judged as false, negating the denoising effect of the denoiser.

[0098] Among them, the denoiser uses mean square error as the loss function:

[0099]

[0100] Where, L G represents the denoiser loss function, m represents the logarithm of the combination of the ambient noise slice and the keyboard original electromagnetic signal slice, x i Represents the i-th signal value in the original electromagnetic signal of the keyboard, y i represents the i-th signal value in the keyboard noisy leakage signal, and G represents the mapping established by the denoiser from the keyboard noisy leakage signal to the denoised signal.

[0101] The discriminator adopts the hinge loss function:

[0102]

[0103] Where, L D represents the discriminator loss function; E represents the mathematical expectation; P trepresents the distribution of the keyboard's raw electromagnetic signal, x represents the keyboard's raw electromagnetic signal, g represents the denoised signal, and D is the mapping established by the discriminator, which discriminates between inputs x and g. When the discriminator outputs an output greater than or equal to 1 or less than or equal to -1 for input x or g, it indicates that the discriminator's judgment of x or g is definitely true or false, and the loss function is 0. If the output is between greater than -1 and less than 1, the discriminator's judgment is uncertain, and the loss function is non-zero. Therefore, it is necessary to accumulate the losses to optimize the discriminator parameters and enable the discriminator to more accurately distinguish samples.

[0104] It can be seen from the loss functions of the denoiser and the discriminator that when the probability distributions of the denoised signal and the original electromagnetic signal of the keyboard are infinitely close, the loss function of the denoiser approaches 0; when the probability distribution of the discriminator input x is infinitely close to the distribution of the original electromagnetic signal of the keyboard, and the input g is infinitely far from the distribution of the original electromagnetic signal of the keyboard, the loss function of the discriminator approaches 0.

[0105] Then the loss function of SADAN is:

[0106] L SADAN =L G +L D

[0107] In order to obtain the best denoising effect, the parameter optimization direction of the denoiser is to minimize L G , the parameter optimization direction of the discriminator is to minimize L D .

[0108] In this embodiment, the training method of the SADAN network model includes:

[0109] Step 1: Input the original electromagnetic signal into the discriminator for training and learning its probability distribution;

[0110] Step 2: Input the slices of the original noisy leakage signal and the original electromagnetic signal into the denoiser, and output the denoised signal, which is then input into the discriminator to determine the authenticity;

[0111] Specifically, a random slice of the keyboard's original electromagnetic signal and a random slice of the ambient noise signal are selected, respectively. These two slices are combined to form the original noisy leakage signal. These two slices are then fed into the denoiser as input data and corresponding label data, respectively. After mapping by the denoiser, the denoised signal is output and fed into the discriminator for authenticity determination.

[0112] Step 3: Calculate the loss function value of the SADAN network model, use the gradient descent algorithm to backpropagate the loss error, and update the parameters of the SADAN network model through Adam optimization;

[0113] Step 4: Repeat steps 1-3. When the loss error stabilizes, the SADAN network model training is complete. At this point, the denoiser in the SADAN network model has acquired the ability to "forge" denoised signals, and the discriminator has acquired the ability to distinguish between genuine and fake signals.

[0114] Example 3

[0115] In order to demonstrate the effectiveness of the SADAN network model in denoising keyboard noisy leakage signals, this embodiment first conducts experimental tests and analysis on denoising simulated noisy signals, and proves the effectiveness of the denoising algorithm through the detection and recognition effect of the denoised signals; then, traditional adversarial generative networks and traditional signal denoising methods are added for comparative experiments. Through the comparison of the evaluation indicators of various algorithms, the effectiveness and advantages of the SADAN network model in denoising keyboard noisy leakage signals are more clearly reflected.

[0116] 1. Experimental environment setup and sample set construction

[0117] The experimental environment was a normal indoor environment. The keyboard keystroke electromagnetic signal acquisition device consisted of an electromagnetic signal acquisition caliper and probe, as well as a signal receiver. The caliper model was AH Systems' BCP-620, and the signal probe model was UT-P03. The signal receiver model was a modular NIPXIe-5162 digitizer with 10-bit sampling accuracy. The PS / 2 keyboard model was Lenovo KB-1701, and the USB keyboard model was Dell SK-8115.

[0118] The model training environment in the experiment is deployed based on the cloud server deep learning environment. The cloud server environment is configured on a single NVIDIA Tesla V100 32GB graphics card. The PyTorch deep learning platform is built based on the Linux operating system, with CUDA version 11.0, cuDNN version 8.0, Python version 3.7, and PyTorch version 1.7.

[0119] The sample set required for SADAN network model training includes the keyboard's original electromagnetic signal and a simulated noisy electromagnetic signal. The simulated noisy signal is composed of the keyboard's original electromagnetic signal superimposed with the keyboard's operating environment noise. A caliper connected to the receiver is coupled to the transmission cable of the keyboard in operation. The leakage signal from the keyboard, when no keys are pressed, is collected as the environmental noise component. This signal is then superimposed with the keyboard's original electromagnetic signal to form the simulated noisy signal. This, along with the original electromagnetic signal, serves as the training sample set for the denoiser in the SADAN network model. The discriminator is trained using the original electromagnetic signal.

[0120] Environmental noise from PS / 2 and USB keyboards was collected at sampling rates of 100 k / s and 5 M / s, respectively. The collected electromagnetic environmental noise samples were cut into several environmental noise slices and combined with the original electromagnetic signal to create a simulated noisy signal. The original electromagnetic signal from the keyboard was acquired using a probe connected to the PS / 2 keyboard data cable and the USB keyboard D+ data cable at the same sampling rate. A set of original electromagnetic signal samples was constructed, consisting of the four English letters G, H, K, and L, and the two Arabic numerals 0 and 1. The number of samples constructed corresponded to the number of environmental noise slices.

[0121] 2. Validity Verification

[0122] During the training of the SADAN network model, the epoch is set to 20. Figure 3 This is the decreasing process of the Loss value during training. It can be seen that the discriminator loss value loss_d and the denoiser loss value loss_g steadily decrease with the increase of the training iteration number, and finally converge to a smaller range.

[0123] After training, the denoiser of the SADAN network model outputs a denoised signal, and the discriminator determines the authenticity of the denoised signal. Figure 4 The following figure compares the waveforms of the denoised signal of keyboard key H using the SADAN network model, the original electromagnetic signal of the keyboard, and a simulated noisy signal. It can be seen that after denoising by the SADAN network model, the important peaks in the original electromagnetic signal waveform containing key information are highlighted and enhanced in the denoised signal. The characteristics of each keyboard key signal are hidden in these important peaks. The SADAN network model enhances the useful information in the noisy signal, laying the foundation for subsequent signal recognition using the detection algorithm, and to a certain extent, proves the effectiveness of the algorithm's denoising.

[0124] The keyboard electromagnetic signals before and after denoising were fed into the GRU-CN recognition model as training samples for key signal recognition. Table 1 shows the comparative results. As can be seen, the GRU-CN detection and recognition algorithm performs better on the denoised signals than on the undenoised signals. This is because the SADAN network model successfully suppresses the noise in the noisy signals, preserving the valid signal. Because the noise is weaker, the valid information is indirectly enhanced, allowing the GRU-CN to more efficiently and accurately extract signal features for detection and recognition.

[0125] Table 1

[0126]

[0127] In order to more intuitively demonstrate the denoising effect of SADAN, this embodiment adds a traditional GAN ​​network to compare its denoising capabilities as an adversarial generative network, and also adds two traditional signal denoising methods, EMD and bandpass filter, for comparison. The discriminator and denoiser of the traditional GAN ​​network both use convolutional neural networks as the network framework. The EMD method is generally used to analyze non-stationary signals. It can process a variety of complex signals, such as biological signals, seismic signals, etc. The EMD decomposition process is a way to convert non-stationary signals into stationary signals. It does not require setting any basis functions, but decomposes according to the characteristics of the signal itself. Its principle is to decompose the fluctuations and trends of different sizes in the signal layer by layer to obtain a series of data sequences with different feature sizes. These data sequences are called eigenmodes. Each eigenmode is a locally stationary sine wave, and its instantaneous frequency and amplitude can be obtained using the Hilbert transform. In this embodiment, the noisy signal is decomposed into four eigenmodes, and the sum of 2 and 3 is output as the denoising result. A bandpass filter is a filter that only allows specific frequencies to pass through, while effectively suppressing signals of other frequencies. This selectivity for specific frequency signals makes it widely used in the field of signal denoising.

[0128] The root mean square error (RMSE) is used as the first evaluation metric for the denoising method. RMSE is often used to measure the deviation between the observed value and the true value. The calculation process uses the denoised signal as the observed value and the original keyboard electromagnetic signal as the true value. The formula is as follows:

[0129]

[0130] The signal-to-noise ratio calculated using the denoised signal and the original signal is used as the second evaluation indicator. The formula is as follows:

[0131]

[0132] Among them, x represents the original electromagnetic signal of the keyboard, x i represents the i-th signal value in the original electromagnetic signal of the keyboard, g represents the denoised signal, and N is the length of the signal slice, that is, the number of sampling points.

[0133] RMSE reflects the difference between the denoised signal and the original signal, indicating the algorithm's ability to protect the effective information in the leaked signal. The smaller the RMSE, the less effective information is lost during the denoising process. SNR is used to reflect the denoising effect of the algorithm. The larger the SNR, the more obvious the denoising effect. The comparison results are shown in Table 2:

[0134] Table 2

[0135]

[0136] By simulating a noisy signal and adding the original keyboard electromagnetic signal, the initial SNR was calculated to be -2.673dB. The RMSE and SNR of the denoising results of the four algorithms were calculated, as shown in Table 2. The results show that SADAN achieved the lowest RMSE and the highest SNR compared to GAN, EMD, and bandpass filters for both evaluation metrics. This demonstrates that SADAN performs well both in protecting the effective information in the simulated noisy keyboard signal and suppressing noise. Because GAN lacks a self-attention layer, the denoiser is unable to accurately capture the long-range relationships between sample points in the keyboard electromagnetic leakage signal sequence during learning, resulting in poor denoising results. EMD and bandpass filters, on the other hand, lack a neural network to extract features from the signal, and therefore cannot accurately grasp the characteristics of the original keyboard leakage signal, thus failing to protect the effective information in the signal.

[0137] Example 4

[0138] In order to verify the practical application value of the SADAN algorithm, this embodiment carries out a keyboard leakage signal denoising experiment.

[0139] The experimental environment setup is consistent with the simulation experiment, with some differences in the sample set. Since USB keyboards lack auto-repeat and auto-repeat delay mechanisms in hardware, in order to obtain sufficient data for model training, it is necessary to continuously tap the USB keyboard keys to continuously emit electromagnetic signals carrying keystroke information. The collected leakage signals are then subjected to endpoint detection and cut into sample slices containing keystroke leakage information, thereby constructing a sufficient training sample set. SADAN's discriminator still uses the original keyboard electromagnetic signals as training samples; the denoiser uses a combination of keyboard leakage signals and the original keyboard electromagnetic signals, with the keyboard leakage signals as input data and the original signals as corresponding label data for training. Figure 5 The waveform comparison of the leakage signal of button H and the original electromagnetic signal shows that under the influence of the complex electromagnetic environment, there is a significant gap between the original electromagnetic signal and the leakage signal.

[0140] The sample set of the keyboard's original electromagnetic signal is constructed in the same way as the simulation experiment. The keyboard leakage signal is obtained by coupling the receiver with the transmission cable of the USB keyboard in the key state through a caliper. The number of sample sets is also the same as the simulation experiment. The validity and practical significance of the SADAN network model are verified by using the actual keyboard leakage signal collected in a complex electromagnetic environment. The results are shown in the figure below. Figure 6 As shown in the figure, after SADAN denoising, the important peak part containing key information is enhanced in the denoised signal, indicating that the effective information in the leakage signal is strengthened and the ambient noise is suppressed.

[0141] Similar to the simulation experiment, the keyboard leakage signals before and after denoising were used as the training set for the GRU-CN recognition model. Leakage signal detection and recognition tests were conducted, yielding the results shown in Table 3. As can be seen, the GRU-CN detection and recognition algorithm demonstrates a certain level of recognition capability for the two keyboard electromagnetic leakage signals after denoising, with significant improvement compared to the pre-denoising levels. This demonstrates that the SADAN network model possesses a certain degree of denoising capability even in complex electromagnetic environments.

[0142] Table 3

[0143]

[0144] The comparative experiments are consistent with the simulation experiments, comparing the GAN, EMD, and bandpass filter algorithms, and using RMSE and SNR as evaluation metrics. The results in Table 4 show that SADAN achieves the best retention of effective information in denoising keyboard leakage signals, resulting in the greatest improvement in the signal-to-noise ratio of the denoised signal.

[0145] Table 4

[0146]

[0147] In summary, the denoising ability of the SADAN algorithm was tested using keyboard leakage signals measured in a complex electromagnetic environment. The results show that the SADAN algorithm has certain practical denoising capabilities.

[0148] In order to restore the keyboard usage in real working scenarios, this embodiment designs a long text keyboard electromagnetic information recognition test. The experimenter uses the test keyboard to input at a normal typing frequency. Figure 7 The text content in , a total of 205 key information.

[0149] The electromagnetic leakage signal emitted by the keyboard during operation is intercepted by a caliper. The leakage signal sample is first denoised using the SADAN algorithm. The denoised signal is then input into the trained recognition model to identify key information. The results are shown in Table 5:

[0150] Table 5

[0151]

[0152] Through comparative experiments, it can be seen that other methods cannot achieve a high recognition accuracy for keyboard electromagnetic leakage signals in complex electromagnetic environments. The combination of SADAN and GRU-CN achieves the best recognition results, indicating that the denoising algorithm combined with the detection algorithm proposed in the present invention also has certain detection and recognition capabilities for electromagnetic leakage signals during continuous text input on the keyboard, reflecting the advanced nature of the detection and recognition algorithm of the present invention.

[0153] The embodiments described above are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by persons skilled in the art should fall within the scope of protection defined by the claims of the present invention.

Claims

1. A keyboard electromagnetic leakage signal noise reduction method based on generative adversarial network, characterized in that: The following steps are involved: S1. Collect the original noisy leakage signal of the keyboard and generate the data to be denoised; The original noisy leakage signal includes: an original electromagnetic signal and an environmental noise signal; S2. Construct a SADAN network model, and perform denoising on the data to be denoised based on the SADAN network model to complete the noise reduction processing of the keyboard electromagnetic leakage signal; The SADAN network model includes: a denoiser and a discriminator; The denoiser is composed of a convolution layer, a self-attention layer, a deconvolution layer and a ReLU activation layer; the denoiser is used to denoise the data to be denoised to obtain a denoised signal; The discriminator is composed of a convolutional layer, a self-attention layer, and a LeakyReLU activation layer; the discriminator is used to judge the authenticity of the denoised signal and output the denoised signal judged to be true; The denoiser uses mean square error as the loss function: Where, represents the denoiser loss function, represents the logarithm of the combination of the ambient noise slice and the keyboard's original electromagnetic signal slice, Indicates the first signal values, y i Indicates the first signal values, represents the mapping established by the denoiser from the keyboard noisy leakage signal to the denoised signal; The discriminator adopts the hinge loss function: Where, represents the discriminator loss function; E represents the mathematical expectation; Indicates the distribution of the original electromagnetic signal of the keyboard, x Indicates the original electromagnetic signal of the keyboard, represents the denoised signal, The mapping established for the discriminator is x and To identify.

2. The keyboard electromagnetic leakage signal noise reduction method based on generative adversarial network according to claim 1 is characterized in that: The training method of the SADAN network model includes: S21, inputting the original electromagnetic signal into the discriminator for training and learning; S22, inputting the slices of the original noisy leakage signal and the original electromagnetic signal into a denoiser, outputting the denoised signal, and inputting the denoised signal into the discriminator to determine authenticity; S23, calculating the loss function value of the SADAN network model, and back-propagating the loss error to optimize and update the parameters of the SADAN network model; S24. Repeat S21-S23. When the loss error is stable, the SADAN network model training is completed.

3. A keyboard electromagnetic leakage signal noise reduction system based on generative adversarial networks, characterized by: include: Acquisition module and denoising module; The acquisition module is used to collect the original noisy leakage signal of the keyboard and generate the data to be denoised; The original noisy leakage signal includes: an original electromagnetic signal and an environmental noise signal; The denoising module is used to construct a SADAN network model and denoise the data to be denoised based on the SADAN network model to complete the noise reduction processing of the keyboard electromagnetic leakage signal; The SADAN network model includes: a denoiser and a discriminator; The denoiser is composed of a convolution layer, a self-attention layer, a deconvolution layer and a ReLU activation layer; the denoiser is used to denoise the data to be denoised to obtain a denoised signal; The discriminator is composed of a convolutional layer, a self-attention layer, and a LeakyReLU activation layer; the discriminator is used to judge the authenticity of the denoised signal and output the denoised signal judged to be true; The denoiser uses mean square error as the loss function: Where, represents the denoiser loss function, represents the logarithm of the combination of the ambient noise slice and the keyboard's original electromagnetic signal slice, Indicates the first signal values, y i Indicates the first signal values, represents the mapping established by the denoiser from the keyboard noisy leakage signal to the denoised signal; The discriminator adopts the hinge loss function: Where, represents the discriminator loss function; E represents the mathematical expectation; Indicates the distribution of the original electromagnetic signal of the keyboard, x Indicates the original electromagnetic signal of the keyboard, represents the denoised signal, The mapping established for the discriminator is x and To identify.

4. The keyboard electromagnetic leakage signal noise reduction system based on generative adversarial network according to claim 3 is characterized in that: The method for training the SADAN network model by the denoising module includes: Step 1: inputting the original electromagnetic signal into the discriminator for training and learning; Step 2: Inputting the slices of the original noisy leakage signal and the original electromagnetic signal into a denoiser, outputting the denoised signal, and inputting the denoised signal into the discriminator to determine authenticity; Step 3: Calculate the loss function value of the SADAN network model, and back-propagate the loss error to optimize and update the parameters of the SADAN network model; Step 4: Repeat steps 1 to 3. When the loss error is stable, the SADAN network model training is completed.

Images