Electromagnetic leakage restoration image super-resolution reconstruction method based on contrastive learning

Abstract

The application discloses a method for electromagnetic leakage restoration image super-resolution reconstruction based on contrast learning, comprising the following steps: building an electromagnetic leakage signal data acquisition platform, acquiring electromagnetic leakage signals, processing the acquired electromagnetic leakage signals to obtain low-resolution images, obtaining a high-resolution reference image by directly acquiring a screenshot image from a display device for the same display content, and constructing a training data set based on paired low-resolution-high-resolution training samples; constructing a super-resolution network, wherein the super-resolution network comprises an input layer, a feature extraction layer, an up-sampling layer and an output layer, and the feature extraction layer is composed of multiple MSRB modules; calculating a contrast loss and a super-resolution reconstruction loss based on the training data set, training the super-resolution network; and performing super-resolution reconstruction by using the trained super-resolution network. The application can better generalize to a real electromagnetic leakage restoration scene, has strong anti-interference capability, and realizes full-automatic enhancement from a physical signal to a high-quality image.

Description

Technical Field

[0001] This invention relates to the field of electromagnetic leakage signal restoration technology, and in particular to a super-resolution reconstruction method for electromagnetic leakage restoration images based on contrastive learning. Background Technology

[0002] Computer display devices, such as monitors and cables, generate varying degrees of electromagnetic leakage during operation. Specific signal receiving equipment and signal processing techniques can intercept these electromagnetic leakage signals and reconstruct the displayed content. Electromagnetic leakage image reconstruction (EMS) is a technique that reconstructs a high-resolution image from a low-resolution image obtained from electromagnetic leakage signals from a computer display device. This technology aims to extract useful information from the leakage signal, improving image quality while enhancing information security. It represents a key research direction for the deep integration of electromagnetic information security and image processing.

[0003] Currently, some progress has been made in the reconstruction of electromagnetic leakage signals based on TEMPEST technology. gr-tempest can capture electromagnetic leakage signals from display connections such as HDMI cables using software-defined radio (SDR) devices and initially reconstruct the displayed image. Furthermore, existing research has proposed methods for recovering electromagnetic features from computer images based on encoder-decoder mapping models, as well as methods for recovering electromagnetic leakage information based on wavelet analysis and inter-frame staggered image fusion algorithms. However, these methods suffer from low resolution and poor image quality in the reconstructed electromagnetic leakage signals. Additionally, due to the lack of real-world training datasets and susceptibility to random noise, existing super-resolution methods are not suitable for real-world electromagnetic leakage scenarios, resulting in low-quality reconstructed images. Summary of the Invention

[0004] To overcome the aforementioned problems in the existing technology, this invention proposes a super-resolution reconstruction method for electromagnetic leakage restoration images based on contrastive learning.

[0005] The technical solution adopted by this invention to solve its technical problem is: a super-resolution reconstruction method for electromagnetic leakage restoration images based on contrastive learning, comprising the following steps; Step 1: Build an electromagnetic leakage signal data acquisition platform to acquire electromagnetic leakage signals. Process the acquired electromagnetic leakage signals to obtain low-resolution images. For the same display content, directly acquire screenshot images from the display device to obtain high-resolution reference images. Based on paired low-resolution-high-resolution training samples, construct a training dataset. Step 2: Construct a super-resolution network, which includes an input layer, a feature extraction layer, an upsampling layer, and an output layer. The feature extraction layer consists of multiple MSRB modules. Step 3: Based on the training dataset obtained in Step 1, calculate the contrast loss and super-resolution reconstruction loss, and train the super-resolution network from Step 2. Step 4: Use the super-resolution network trained in Step 3 to perform super-resolution reconstruction.

[0006] The above-mentioned electromagnetic leakage reconstruction image super-resolution method based on contrastive learning, specifically step 1, involves: constructing an electromagnetic leakage signal acquisition platform to capture electromagnetic leakage signals unintentionally radiated by computer display devices during operation; sequentially filtering, adjusting the sampling rate, extracting synchronization signals, and reconstructing image data based on gr-tempest to obtain low-resolution images; for the same display content, directly acquiring screenshot images from the display device to obtain corresponding high-resolution reference images, forming pairs of low-resolution-high-resolution training samples, and completing the construction of a dataset under real electromagnetic leakage scenarios.

[0007] The super-resolution reconstruction method for electromagnetic leakage restoration images based on contrastive learning described above, in step 2, the MSRB module includes convolutional layers and bidirectional LSTM, which are used to extract deep feature maps from the input image.

[0008] The above-mentioned electromagnetic leakage reconstruction image super-resolution method based on contrastive learning, specifically includes step 3 as follows: Step 3.1: Input the low-resolution input image into the super-resolution network, and take the output of the feature extraction module as the low-resolution image feature F. L The high-resolution target image is input into the super-resolution network, and the output of the feature extraction module is taken as the high-resolution image feature F. H ; Step 3.2: For N samples in a batch, extract the low-resolution image features F respectively. L and high-resolution image features F H Perform normalization; Step 3.3, calculate the low-resolution image features F L and high-resolution image features F H The similarity matrix between them is used to generate a label sequence labels=[0,1,2,…,N-1]; Step 3.4: Calculate the contrast loss and super-resolution reconstruction loss respectively, and construct the total loss function L. tot : L tot =L re +α*L con Where α is the weighting coefficient, L re For super-resolution reconstruction loss, L con To compare the losses.

[0009] The super-resolution reconstruction loss in the above-mentioned electromagnetic leakage reconstruction image super-resolution reconstruction method based on contrastive learning is specifically as follows: L re =||I SR -I HR ||2 2 Among them, I SR For super-resolution image reconstruction, I HR It is a high-resolution, realistic image.

[0010] The contrast loss in the above-mentioned electromagnetic leakage reconstruction image super-resolution method based on contrastive learning is specifically: L con =(CE(S,labels)+CE(S T ,labels)) / 2 Where CE is the cross-entropy loss function, labels is the label sequence, and S is the similarity matrix between low-resolution features and high-resolution features.

[0011] The beneficial effects of the present invention are: (1) It can better generalize to real electromagnetic leakage restoration scenarios. The present invention builds a dedicated electromagnetic leakage signal acquisition platform to collect electromagnetic leakage signals in real scenarios and restore them to low-resolution images. At the same time, it obtains the corresponding high-resolution reference images, forming a dedicated training dataset for electromagnetic leakage scenarios. This enables the super-resolution model trained on this dataset to better generalize to real electromagnetic leakage restoration scenarios.

[0012] (2) Strong anti-interference ability. A super-resolution reconstruction technique for electromagnetic leakage restoration images based on contrastive learning is proposed, which enables the model to ignore pixel-level random noise disturbances, thereby improving the quality of the reconstructed image.

[0013] (3) Complete end-to-end process. A complete closed loop is formed from the acquisition of electromagnetic leakage signals to the output of high-resolution images, realizing fully automatic enhancement from physical signals to high-quality images. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the process of this invention; Figure 2 This is a schematic diagram of the super-resolution network of the present invention; Figure 3 This is a schematic diagram of the MSRB module of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] This embodiment addresses the lack of datasets for real-world electromagnetic leakage scenarios by constructing an electromagnetic leakage signal data acquisition platform to build a dataset of real-world electromagnetic leakage scenarios for training the super-resolution reconstruction network. Electromagnetic leakage reconstructed images often exhibit pixel-level offsets, a problem that existing super-resolution algorithms using pixel-by-pixel loss functions struggle to solve. Therefore, this invention introduces contrastive learning, applying contrastive constraints in the feature space. Through image-level overall feature similarity calculation, the feature extractor ignores pixel-level random noise perturbations, enabling effective training without relying on precise alignment.

[0017] like Figure 1 As shown, the electromagnetic leakage reconstruction image super-resolution reconstruction method based on contrastive learning in this embodiment specifically includes the following steps: Step 1: Build an electromagnetic leakage signal data acquisition platform to acquire electromagnetic leakage signals. Process the acquired electromagnetic leakage signals to obtain low-resolution images. For the same display content, directly acquire screenshot images from the display device to obtain high-resolution reference images. Based on paired low-resolution and high-resolution training samples, construct a training dataset.

[0018] An electromagnetic leakage signal acquisition platform is constructed using antennas, signal receiving boards, software-defined radio devices, and computer monitors to capture electromagnetic leakage signals unintentionally radiated by computer display devices (including but not limited to monitors, HDMI cables, VGA cables, etc.) during operation. Based on gr-tempest, the acquired electromagnetic leakage signals are sequentially filtered, sample rate adjusted, synchronization signal extracted, and image data reconstructed to obtain low-resolution images. For the same displayed content, screenshots are directly acquired from the display device to obtain corresponding high-resolution reference images, forming pairs of low-resolution and high-resolution training samples. This completes the construction of a dataset for real-world electromagnetic leakage scenarios, used for subsequent training of super-resolution network models.

[0019] Step 2: Construct a super-resolution network, which includes an input layer, a feature extraction layer, an upsampling layer, and an output layer, such as... Figure 2 As shown, the feature extraction layer consists of multiple MSRB modules, used to extract deep feature maps from the input image. Each MSRB module comprises a convolutional layer and a bidirectional LSTM, with the specific structure as follows: Figure 3 As shown.

[0020] Step 3: Based on the training dataset obtained in Step 1, calculate the contrast loss and super-resolution reconstruction loss, and train the super-resolution network from Step 2.

[0021] Step 3 specifically includes: Step 3.1: Input the low-resolution input image into the super-resolution network, and take the output of the feature extraction module as the low-resolution image feature F.L The high-resolution target image is input into the super-resolution network, and the output of the feature extraction module is taken as the high-resolution image feature F. H .

[0022] Step 3.2: For N samples in a batch, extract the low-resolution image features F respectively. L and high-resolution image features F H Normalize: F Lnorm =normalize(F L ), F Hnorm =normalize(F H ).

[0023] Step 3.3, calculate the low-resolution image features F L and high-resolution image features F H The similarity matrix between them is S(i,j)=flatten(F Lnorm,i )*flatten(F Hnorm,j ) T , where S(i,j) represents the cosine similarity between the i-th low-resolution image feature and the j-th high-resolution image feature. Generate a label sequence labels=[0,1,2,…,N-1], that is, only when i=j, the i-th low-resolution image and the j-th high-resolution image constitute a positive sample pair, and the rest are negative sample pairs.

[0024] Step 3.4: Calculate the contrast loss and super-resolution reconstruction loss respectively, and construct the total loss function L. tot : L tot =L re +α*L con Where α is the weighting coefficient, L re For super-resolution reconstruction loss, L con To compare the losses.

[0025] The super-resolution reconstruction loss is the L2 loss, used to constrain the super-resolution reconstructed image I. SR With high-resolution real images I HR For pixel-level consistency, the super-resolution reconstruction loss is specifically as follows: L re =||I SR -I HR ||2 2 Among them, I SR For super-resolution image reconstruction, I HR It is a high-resolution, realistic image.

[0026] The specific comparative loss is as follows: L con =(CE(S,labels)+CE(S T ,labels)) / 2 Where CE is the cross-entropy loss function, labels is the label sequence, and S is the similarity matrix between low-resolution features and high-resolution features.

[0027] Step 4: Use the super-resolution network trained in Step 3 to perform super-resolution reconstruction.

[0028] This invention proposes a super-resolution reconstruction method for electromagnetic leakage reconstruction images based on contrastive learning, which can achieve high-resolution reconstruction of electromagnetic leakage reconstruction images.

[0029] This invention constructs a dedicated electromagnetic leakage signal data acquisition platform and training dataset. By building a dedicated electromagnetic leakage signal acquisition platform, this invention collects electromagnetic leakage signals in real-world scenarios and reconstructs them into low-resolution images, while simultaneously acquiring corresponding high-resolution reference images, thus forming a dedicated training dataset for electromagnetic leakage scenarios.

[0030] This invention proposes a super-resolution reconstruction technique for electromagnetic leakage restoration images based on contrastive learning. By contrastive learning of features from high- and low-resolution images, the model can ignore pixel-level random noise disturbances and focus on essential features that characterize the content and structure of the image, thereby improving the quality of the reconstructed image.

[0031] The above embodiments are merely exemplary embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art can make various modifications or equivalent substitutions to the present invention within its scope and spirit, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of the present invention.

Claims

1. A method for super-resolution reconstruction of electromagnetic leakage images based on contrastive learning, characterized in that, Includes the following steps; Step 1: Build an electromagnetic leakage signal data acquisition platform to acquire electromagnetic leakage signals. Process the acquired electromagnetic leakage signals to obtain low-resolution images. For the same display content, directly acquire screenshot images from the display device to obtain high-resolution reference images. Based on paired low-resolution-high-resolution training samples, construct a training dataset. Step 2: Construct a super-resolution network, which includes an input layer, a feature extraction layer, an upsampling layer, and an output layer. The feature extraction layer consists of multiple MSRB modules. Step 3: Based on the training dataset obtained in Step 1, calculate the contrast loss and super-resolution reconstruction loss, and train the super-resolution network from Step 2. Step 4: Use the super-resolution network trained in Step 3 to perform super-resolution reconstruction.

2. The electromagnetic leakage reconstruction image super-resolution reconstruction method based on contrastive learning according to claim 1, characterized in that, Step 1 specifically involves: constructing an electromagnetic leakage signal acquisition platform to capture electromagnetic leakage signals unintentionally radiated by computer display devices during operation; performing filtering, sampling rate adjustment, synchronization signal extraction, and image data reconstruction on the acquired electromagnetic leakage signals based on gr-tempest to obtain low-resolution images; and directly acquiring screenshot images from the display device for the same display content to obtain corresponding high-resolution reference images, forming pairs of low-resolution and high-resolution training samples to complete the construction of a dataset under real electromagnetic leakage scenarios.

3. The electromagnetic leakage reconstruction image super-resolution reconstruction method based on contrastive learning according to claim 1, characterized in that, In step 2, the MSRB module includes convolutional layers and bidirectional LSTM, which are used to extract deep feature maps from the input image.

4. The electromagnetic leakage reconstruction image super-resolution reconstruction method based on contrastive learning according to claim 1, characterized in that, Step 3 specifically includes: Step 3.1: Input the low-resolution input image into the super-resolution network, and take the output of the feature extraction module as the low-resolution image feature F. L The high-resolution target image is input into the super-resolution network, and the output of the feature extraction module is taken as the high-resolution image feature F. H ; Step 3.2: For N samples in a batch, extract the low-resolution image features F respectively. L and high-resolution image features F H Perform normalization; Step 3.3, calculate the low-resolution image features F L and high-resolution image features F H The similarity matrix between them is used to generate a label sequence labels=[0,1,2,…,N-1]; Step 3.4: Calculate the contrast loss and super-resolution reconstruction loss respectively, and construct the total loss function L. tot : L tot =L re +a*L con Where α is the weighting coefficient, L re For super-resolution reconstruction loss, L con To compare the losses.

5. The electromagnetic leakage reconstruction image super-resolution reconstruction method based on contrastive learning according to claim 4, characterized in that, The super-resolution reconstruction loss is specifically: L re =||I SR -I HR ||2 2 Among them, I SR For super-resolution image reconstruction, I HR It is a high-resolution, realistic image.

6. The electromagnetic leakage reconstruction image super-resolution reconstruction method based on contrastive learning according to claim 4, characterized in that, The comparison loss is specifically as follows: L con =(CE(S,labels)+CE(S T ,labels)) / 2 Where CE is the cross-entropy loss function, labels is the label sequence, and S is the similarity matrix between low-resolution features and high-resolution features.

Images