Electromagnetic information leakage detection and identification method and system for indoor electronic equipment

Abstract

The invention discloses an electromagnetic information leakage detection and identification method and system for indoor electronic equipment, and relates to the technical field of electromagnetic information leakage detection, and the method comprises the steps: obtaining multi-frequency-point signal data of the indoor electronic equipment; obtaining a spectrogram of the signal based on the multi-frequency-point signal data, and inputting the spectrogram into a pre-trained electromagnetic information leakage detection model for detection to obtain an equipment type to which the signal belongs; inputting the multi-frequency-point signal data into a pre-trained double-path feature fusion model for feature extraction to obtain fingerprint features of the signals; and comparing the fingerprint features with individual features pre-stored in a feature library to obtain an equipment individual to which the signal belongs. A comprehensive and comprehensive indoor electronic equipment electromagnetic information leakage detection and identification integrated solution can be provided, different processing methods are carried out on different leakage sources, the electromagnetic information leakage detection and identification accuracy of an existing test instrument is effectively improved, and a technical means is provided for electromagnetic information safety in key fields.

Description

Technical Field

[0001] The present invention relates to the technical field of electromagnetic information leakage detection, and in particular to a method and system for detecting and identifying electromagnetic information leakage of indoor electronic devices. Background Art

[0002] The statements in this section merely provide background technical information related to the present disclosure and do not necessarily constitute prior art.

[0003] The technology for detecting and identifying electromagnetic information leakage of indoor electronic devices is to detect and analyze the leaked electromagnetic signals of various types of indoor office electronic devices, such as computers, monitors, mice, keyboards, printers, etc., so as to obtain the effective information carried by the leaked signals, thereby providing necessary technical means for evaluating the electromagnetic information security of electronic devices. For the method of detecting and identifying electromagnetic information leakage, it is currently mainly concentrated on the detection and restoration of the leakage information of computer monitors. The electromagnetic signals leaked from the monitors are received by a receiver, the field frequency information and line frequency information of the monitors are obtained through autocorrelation calculation, and then the data is rearranged to obtain the picture information displayed on the monitors. There is currently a relatively complete solution for this process, and the technical maturity is relatively high. However, there is currently no effective method to deal with the detection and identification of electromagnetic information leakage of other indoor electronic devices.

[0004] The detection and identification of electromagnetic information leakage from indoor electronic devices involves analyzing and processing the leakage signals to obtain the type of leakage source. At the same time, for the leakage signals, technical means are used to reproduce the leakage information, thus causing electromagnetic information security problems, which is a popular direction in the field of electromagnetic signal processing. Currently, existing technical solutions such as the patent with application number CN202410087599.0, "Method for Detecting and Separating Electromagnetic Information Leakage Signals of Displays Based on Stacked Convolutional Neural Networks", specifically adopt the following method: By performing short-time Fourier transform on the captured electromagnetic signals to obtain time-frequency images, and then building a neural network to extract features from the input images to obtain features suitable for binary classification, so as to determine whether there is a display signal leakage in the current leakage signal. This method starts from image processing, converts the signal feature detection into an image processing method, and classifies the input images through the constructed deep learning network, thereby determining whether there is a leakage source. This invention can meet the requirements for detecting and identifying display leakage information. Another example is the patent with application number CN202011159465.3, "A Fast Detection Method for Electromagnetic Information Leakage Based on Template Matching Technology", whose method is as follows: Through feature engineering, the known leakage signals are subjected to time-frequency conversion to obtain time-frequency diagrams; at the same time, the template matching algorithm is used to extract features from the time-frequency diagrams to obtain the features of the known leakage signals. When an unknown signal is input, first obtain the time-frequency image of the signal, then extract the image features, and determine whether the unknown signal belongs to the display leakage signal through template matching. This method uses the traditional template matching method, and when the template feature calculation is accurate, it can effectively determine whether there is a computer display information leakage.

[0005] The current technical solutions can, to a certain extent, solve the problem of detecting and identifying electromagnetic information leakage from displays. For example, the method for detecting and separating electromagnetic information leakage signals of displays based on stacked convolutional neural networks and a fast detection method for electromagnetic information leakage based on template matching technology can effectively determine the display leakage information by processing the content of the images.

[0006] The prior art starts from the perspective of image processing, converts signal processing into image processing, and then determines whether the leakage signal contains the display leakage signal. However, the existing electromagnetic information leakage detection and identification methods have obvious deficiencies: First, the detection and identification types are single; the prior art focuses on the electromagnetic information leakage detection and processing of computer displays, and there is a technical gap in the electromagnetic information leakage detection and identification methods for common office electronic devices with significantly inconsistent leakage frequency points, such as computers, keyboards, mice, printers, etc. Second, the ability to judge individual leakage sources is significantly insufficient. The prior art is for the detection and identification of electromagnetic information leakage from displays, focusing on solving the problem of the presence or absence of leakage signals, and does not discuss the fingerprint characteristics of the leakage sources themselves. In this way, there is an obvious problem that it is impossible to determine which electronic device the current leakage signal belongs to, thus bringing certain troubles to the electromagnetic information security assessment of electronic devices. Summary of the Invention

[0007] To overcome the above deficiencies of the prior art, the present invention provides a method and system for detecting and identifying electromagnetic information leakage of indoor electronic devices, which can provide a comprehensive and integrated solution for detecting and identifying electromagnetic information leakage of indoor electronic devices, adopt different processing methods for different leakage sources, effectively improve the accuracy of electromagnetic information leakage detection and identification of existing test instruments, and provide technical means for electromagnetic information security in key fields.

[0008] To achieve the above object, one or more embodiments of the present invention provide the following technical solutions:

[0009] In the first aspect, the present invention provides a method for detecting and identifying electromagnetic information leakage of indoor electronic devices, including:

[0010] Obtain multi-frequency point signal data of indoor electronic devices;

[0011] Based on the multi-frequency point signal data, obtain a spectrogram of the signal, and input the spectrogram into a pre-trained electromagnetic information leakage detection model for detection to obtain the device type to which the signal belongs;

[0012] Input the multi-frequency point signal data into a pre-trained dual-channel feature fusion model for feature extraction to obtain the fingerprint feature of the signal; compare the fingerprint feature with the individual features pre-stored in the feature library to obtain the device individual to which the signal belongs;

[0013] The dual-channel feature fusion model sequentially includes a parallel first branch and second branch, and a splicing layer. The first branch sequentially includes a transformation module, a first processing module, a second processing module, a third processing module, a fourth processing module, and a fifth processing module. The second branch sequentially includes an embedding patch module, a position embedding module, a category embedding module, a normalization module, a multi-head attention module, a normalization module, and a multi-layer perceptron.

[0014] A further technical solution is to perform a fast Fourier transform on the multi-frequency point signal data to obtain a spectrogram of the signal.

[0015] A further technical solution is that the electromagnetic information leakage detection model includes a first convolution-batch normalization-nonlinear module, a second convolution-batch normalization-nonlinear module, a first bottleneck module, a second bottleneck module, a maximum pooling layer, and a fully connected layer that are sequentially connected in order.

[0016] A further technical solution is that the first convolution-batch normalization-nonlinear module and the second convolution-batch normalization-nonlinear module have the same structure, and include a 3×3 convolution layer, a batch normalization layer, and an activation function layer that are sequentially connected in order.

[0017] A further technical solution is that the first bottleneck module and the second bottleneck module have the same structure, and include a 1×1 convolution layer, an activation function layer, a 3×3 convolution layer, an activation function layer, a 1×1 convolution layer, a feature fusion layer, and an activation function layer that are sequentially connected in order.

[0018] A further technical solution is that the first processing module sequentially includes a convolution-batch normalization-nonlinear module, a maximum pooling layer, a feature extraction module, and an activation function layer, and the second processing module, the third processing module, the fourth processing module, and the fifth processing module have the same structure, and sequentially include an instance normalization layer, a feature extraction module, and an activation function layer.

[0019] A further technical solution is that the feature extraction module includes a first branch and a second branch. The first branch sequentially includes a 1×1 convolution-batch normalization-nonlinear module, a 3×3 convolution-batch normalization-nonlinear module, and a 1×1 convolution-batch normalization-nonlinear module, and the second branch is a convolution layer.

[0020] In a second aspect, the present invention provides an electromagnetic information leakage detection and identification system for indoor electronic devices, including:

[0021] A signal data acquisition module, which is configured to: acquire multi-frequency point signal data of indoor electronic devices;

[0022] A device type detection module, which is configured to: obtain a spectrogram of the signal based on the multi-frequency point signal data, input the spectrogram into a pre-trained electromagnetic information leakage detection model for detection, and obtain the device type to which the signal belongs;

[0023] A target device recognition module, configured to: input the multi-frequency signal data into a pre-trained dual-channel feature fusion model for feature extraction to obtain fingerprint features of the signal; compare the fingerprint features with the individual features pre-stored in the feature library to obtain the device individual to which the signal belongs.

[0024] The dual-channel feature fusion model sequentially includes a first branch and a second branch arranged in parallel, and a splicing layer. The first branch sequentially includes a transformation module, a first processing module, a second processing module, a third processing module, a fourth processing module, and a fifth processing module. The second branch sequentially includes an embedding patch module, a position embedding module, a category embedding module, a normalization module, a multi-head attention module, a normalization module, and a multi-layer perceptron.

[0025] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps in a method for detecting and identifying electromagnetic information leakage of an indoor electronic device as described in the first aspect are implemented.

[0026] In a fourth aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps in a method for detecting and identifying electromagnetic information leakage of an indoor electronic device as described in the first aspect are implemented.

[0027] The above one or more technical solutions have the following beneficial effects:

[0028] Starting from the leakage signal spectrum information, the present invention determines the type to which the leakage signal source belongs by analyzing the spectrum characteristics of the electromagnetic information leakage signal of the indoor electronic device. At the same time, the time-domain data and frequency-domain feature fusion method is used to extract and identify the individual features of the leakage source, establish a leakage source feature library, and clarify the individual to which the leakage source belongs through feature comparison, making up for the problems of single leakage source detection category, low detection accuracy, and poor individual leakage source recognition ability in the prior art, thereby providing an effective technical means for electromagnetic information security assessment.

[0029] The present invention proposes a method for detecting electromagnetic information leakage of electronic devices based on a spectrogram. By analyzing the spectrogram, it is possible to detect and evaluate the electromagnetic information leakage of common indoor electronic devices, not limited to the detection and identification of the electromagnetic information leakage of the display, effectively improving the accuracy of detecting the electromagnetic information leakage of indoor electronic devices.

[0030] The present invention proposes a method for identifying the fingerprint features of electromagnetic information leakage of electronic devices by time-frequency domain fusion. By fusing time-domain features and frequency-domain features, a fingerprint feature library of the leakage source is established to realize the individual identification of a specific leakage source, effectively improving the usability.

[0031] The present invention can efficiently, quickly, and accurately detect and identify the electromagnetic information leakage of common office electronic devices such as indoor computers, monitors, keyboards, mice, printers, etc.

[0032] In the present invention, all models can be deployed to embedded devices, and the selected detection models are all lightweight models. Therefore, it is particularly suitable for the transformation and improvement of handheld test instruments to improve the level of industrial production automation. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings forming a part of this specification are used to provide a further understanding of the present invention. The schematic embodiments and descriptions thereof of the present invention are used to explain the present invention and do not constitute an improper limitation to the present invention.

[0034] Figure 1 is a flowchart of the detection and identification method of the embodiment of the present invention;

[0035] Figure 2 is a schematic structural diagram of the electromagnetic information leakage detection model of the embodiment of the present invention;

[0036] Figure 3 is a schematic structural diagram of the convolution-batch normalization-nonlinear module network of the embodiment of the present invention;

[0037] Figure 4 is a schematic structural diagram of the bottleneck module network of the embodiment of the present invention;

[0038] Figure 5 is a schematic structural diagram of the dual-path feature fusion model of the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which the present invention belongs.

[0040] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments of the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0041] In the case of no conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0042] Embodiment 1

[0043] As Figure 1As shown in the figure, this embodiment discloses a method for detecting and identifying electromagnetic information leakage of indoor electronic devices, and the method includes the following steps:

[0044] S1: Obtain multi-frequency point signal data of indoor electronic devices;

[0045] In this embodiment, the electromagnetic signals leaked by indoor electronic devices are collected, and a receiver is used to receive signals at different frequency points to obtain multi-frequency point signal data. For indoor electronic devices, common frequency points include 445 MHz, 2.4 GHz and other frequency points.

[0046] By collecting signal data from known leakage sources, training data is constructed. For example, a signal receiver is used to collect the electromagnetic signals of common indoor monitors, keyboards, mice, etc., and each type of signal is marked with a category. At the same time, for devices of the same type, such as monitors, the electromagnetic signals leaked by different monitors are further tagged to distinguish each monitor. This process is similar to face recognition, where data is collected for each face. Using the same idea, data is collected for each individual monitor to create a data set belonging to that individual monitor; the electromagnetic information leakage detection model is used to distinguish whether the leakage signal belongs to a monitor or other devices such as keyboards, and the dual-channel feature fusion model is used to distinguish which monitor or which keyboard it is. During the signal collection process, 10,000-20,000 data sets are collected for each large category, and 10-20 data are collected for each individual in each large category.

[0047] S2: Obtain a spectrogram of the signal based on the multi-frequency point signal data, and input the spectrogram into a pre-trained electromagnetic information leakage detection model for detection to obtain the device type to which the signal belongs;

[0048] In this embodiment, considering that there may be interference signals such as wifi signals indoors, electromagnetic information leakage detection based on spectrograms is used at different frequency points to determine the current leakage information, that is, the device type to which the obtained electromagnetic signal data belongs.

[0049] Perform a fast Fourier transform on the multi-frequency point signal data to obtain a spectrogram of the signal.

[0050] As Figure 2 shown, the electromagnetic information leakage detection model includes a first convolution-batch normalization-nonlinear module, a second convolution-batch normalization-nonlinear module, a first bottleneck module, a second bottleneck module, a maximum pooling layer, and a fully connected layer that are sequentially connected in order.

[0051] As Figure 3As shown in the figure, the first Convolution-Batch Normalization-Nonlinearity module (CBR module 1) and the second Convolution-Batch Normalization-Nonlinearity module (CBR module 2) have the same structure, both of which are Convolution-Batch Normalization-Nonlinearity modules, including a 3×3 convolutional layer, a Batch Normalization layer Batch Normhe, and a ReLU activation function layer connected in sequence. The CBR module is used to extract feature information from the spectrogram. As the network structure deepens, the extracted features tend to be more high-dimensional semantic features, so that the high-dimensional features of the data can be used to classify the types of leakage sources and improve the classification accuracy.

[0052] As Figure 4 shown in the figure, the first Bottleneck module (BottleNeck module 1) and the second Bottleneck module (BottleNeck module 2) have the same structure, both of which are Bottleneck modules, including a 1×1 convolutional layer, a ReLU activation function layer, a 3×3 convolutional layer, a ReLU activation function layer, a 1×1 convolutional layer, a Feature Fusion layer Add, and a ReLU activation function layer connected in sequence. The Bottleneck module is designed in a residual form, that is, the input of the module is input to the Add operator in a direct connection form, which is a shortcut form of skip connection to achieve deep and shallow layer feature fusion. By fusing deep and shallow layer features, the feature expression ability of the model is effectively improved, and at the same time, it is convenient for the training and inference of the neural network model and improves the inference speed.

[0053] Obvious spikes and relatively wide burst transmission signals can be seen from the spectrogram of electromagnetic leakage. Therefore, when performing leakage analysis, more attention is paid to the edge information of the spectrogram line. For this reason, the present invention uses maximum pooling to retain edge features.

[0054] The spectrogram is input into the electromagnetic information leakage detection model for detection. By using the powerful feature extraction ability of deep learning, feature information at different levels of the spectrogram is obtained. At the same time, by using the method of deep and shallow layer feature fusion, the sensitivity of leakage signal feature detection is further improved.

[0055] S3: Input the multi-frequency point signal data into a pre-trained dual-channel feature fusion model for feature extraction to obtain the fingerprint features of the signal; compare the fingerprint features with the individual features pre-stored in the feature library to obtain the device individual to which the signal belongs.

[0056] In this embodiment, as Figure 5 shown in the figure, a dual-channel feature fusion model is constructed and trained. This model is a dual-channel feature fusion network structure. One channel processes the time-frequency diagram. By performing short-time Fourier transform on the multi-frequency point signal data (IQ data), the time-frequency diagram is obtained, and then through 5 stages of processing modules, frequency domain feature information is obtained; the other channel directly processes the multi-frequency point signal data (IQ data). By introducing a multi-head attention mechanism, the feature extraction ability of the network for time series signals is effectively improved.

[0057] The dual-path feature fusion model successively includes a first path branch and a second path branch in parallel, and a concatenation layer Concat. The first path branch successively includes a transformation module, a first processing module, a second processing module, a third processing module, a fourth processing module, and a fifth processing module. Each processing module belongs to the corresponding stage, namely stage 1 to stage 5. The first path branch focuses on extracting the features of the time-frequency diagram, that is, extracting the individual features of the leakage source from the perspective of the image. The second path branch directly performs time-series processing from the perspective of IQ data, focusing on solving the extraction of effective features in the long time-series process of the signal. The concatenation layer is used to concatenate the features obtained by the two path branches.

[0058] The transformation module is a short-time Fourier transform module STFT, which is used to perform a short-time Fourier transform on the multi-frequency point signal data (IQ data) to obtain a time-frequency diagram.

[0059] The first processing module successively includes a convolution-batch normalization-nonlinearity module, a max-pooling layer, a feature extraction module, and a ReLU activation function layer; the second processing module, the third processing module, the fourth processing module, and the fifth processing module have the same structure and successively include an instance normalization layer IN, a feature extraction module, and a ReLU activation function layer. Further, the feature extraction module includes a first branch and a second branch. The first branch successively includes a 1×1 convolution-batch normalization-nonlinearity module, a 3×3 convolution-batch normalization-nonlinearity module, and a 1×1 convolution-batch normalization-nonlinearity module. The second branch is a convolutional layer. Among them, the 1×1 convolution-batch normalization-nonlinearity module (CBR1×1) is a convolution-batch normalization-nonlinearity module with a convolution kernel size of 1×1, and the 3×3 convolution-batch normalization-nonlinearity module (CBR3×3) is a convolution-batch normalization-nonlinearity module with a convolution kernel size of 3×3.

[0060] The instance normalization layer IN is a normalization technique that independently calculates the mean and variance on each channel of each sample and then normalizes these channels. It stabilizes the training process by allowing each channel to be adjusted independently.

[0061] When the multi-frequency signal data is processed in the first branch, first perform a short-time Fourier transform (STFT) on the multi-frequency signal data to obtain the time-frequency diagram of the signal data, and input the time-frequency diagram into the first branch for processing to obtain frequency domain features, that is, the high-dimensional semantic features in the image. Specifically, the feature extraction process is as follows: (1) Initial convolution: The input image first passes through a CBR layer to extract preliminary low-level features. (2) Downsampling: The max pooling layer reduces the size of the feature map, retains important features, and reduces the computational amount at the same time. (3) Depth convolution: 1×1 convolution is used to adjust the number of channels, 3×3 convolution is used to extract spatial features, and then 1×1 convolution is used to adjust the number of channels. (4) Nonlinear activation: ReLU introduces nonlinearity to enable the network to learn complex patterns. (5) Normalization: Instance normalization helps to stabilize and accelerate training. (6) Repetition stage: Each subsequent stage further processes the features to extract more abstract representations.

[0062] The second branch sequentially includes an Embedded Patches module, a Positional Embedding module, a Class Embedding module, a Norm module, a Multi-head Attention module, a Norm module, and a Multi-Layer Perceptron (MLP).

[0063] Input the multi-frequency signal data into the second branch for feature extraction to obtain time domain features. Specifically, the feature extraction process is as follows: The input IQ data is stacked according to the time series and then split into multiple patches, and each patch is embedded as a vector; positional embedding is added to retain positional information; class embedding is inserted for classification tasks; the normalization layer ensures data stability; the multi-head attention mechanism allows the model to focus on different patch positions; another normalization layer further stabilizes the data; the multi-layer perceptron processes the final feature representation for classification or other tasks. Through these steps, the model can effectively extract and process features from the input data and finally be used for various downstream tasks.

[0064] Input the frequency domain features and the time domain features into the concatenation layer for feature fusion, and obtain the final fingerprint features by stacking the features. This makes up for the reduction in recognition rate caused by the possible incomplete feature representation in single-dimensional feature extraction.

[0065] Similar to the face recognition process, during the training process of the dual-path feature fusion model, the radiation electromagnetic signals of different devices such as monitors, keyboards, mice, and printers and the corresponding categories are used as labels for training to train the feature extraction ability of the model; in the verification stage, feature comparison is performed by stacking the obtained fingerprint features, and the cosine similarity is used to determine the device category to which the unknown signal belongs, realizing the fingerprint feature recognition of electronic devices.

[0066] The acquired fingerprint features are compared with the individual features corresponding to the device individuals pre-stored in the feature library, and a specific threshold is set. When the comparison result is lower than the specific threshold (the threshold in this embodiment is 0.6), the current fingerprint features are determined to belong to the newly added device individual, and the feature library is updated immediately. After the comparison is completed, different information restoration algorithms are executed for different device types to achieve detection and restoration of leaked information.

[0067] Furthermore, the construction and update of the feature library both refer to adding or updating individual feature vectors. The feature library is composed of individual feature vectors. When an unknown individual appears, the feature vector of the individual is added to the feature library to update the feature library. Specifically, the feature library construction method: first, when the device type and device individual are known, the aforementioned two-way feature fusion model is used to extract features, and the feature values ​​in the form of column vectors are obtained as features in the feature library. This process is the feature library construction process. The feature library update process refers to the process in which, in an unknown environment, the identified device under test is found through feature comparison to not belong to any feature (device individual) in the feature library. The feature is automatically entered into the library as a new device individual feature for updating the feature library. In this way, when the device is detected for the second time, the device individual can be determined through feature comparison.

[0068] Furthermore, the feature comparison uses cosine similarity, which is expressed as:

[0069]

[0070] Where a is the unknown signal feature value, b is the individual feature of a leakage source in the feature library, n represents the individual feature dimension, such as 256 or 512, k represents the kth feature value index of the n-dimensional feature, and x 1k represents the kth eigenvalue of the unknown signal feature a, x 2k Represents the kth eigenvalue of an individual feature in the b signal feature library.

[0071] Traverse all the features in the feature library and take the highest similarity value. If the highest value is higher than 0.6, the unknown signal belongs to the individual with the highest similarity in the feature library. If it is lower than 0.6, it is a newly added unknown individual. At this time, feature a is added to the library and marked as a newly added leakage source individual.

[0072] Embodiment 2

[0073] This embodiment discloses an electromagnetic information leakage detection and identification system for indoor electronic equipment, including:

[0074] A signal data acquisition module is configured to: acquire multi-frequency signal data of indoor electronic equipment;

[0075] The device type detection module is configured to: obtain a spectrogram of the signal based on the multi-frequency point signal data, input the spectrogram into a pre-trained electromagnetic information leakage detection model for detection, and obtain the device type to which the signal belongs;

[0076] The target device identification module is configured to: input the multi-frequency point signal data into a pre-trained dual-channel feature fusion model for feature extraction to obtain the fingerprint feature of the signal; compare the fingerprint feature with the individual features pre-stored in the feature library to obtain the device individual to which the signal belongs;

[0077] The dual-channel feature fusion model sequentially includes a first branch and a second branch arranged in parallel, and a splicing layer. The first branch sequentially includes a transformation module, a first processing module, a second processing module, a third processing module, a fourth processing module, and a fifth processing module. The second branch sequentially includes an embedding patch module, a position embedding module, a category embedding module, a normalization module, a multi-head attention module, a normalization module, and a multi-layer perceptron.

[0078] Embodiment III

[0079] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method in Embodiment I are implemented.

[0080] Embodiment IV

[0081] The purpose of this embodiment is to provide a computer-readable storage medium. A computer-readable storage medium stores a computer program, and when the program is executed by a processor, the steps of the method in Embodiment I are executed.

[0082] The steps involved in the devices in the above Embodiments III and IV correspond to those in Method Embodiment I. For the specific implementation manners, reference may be made to the relevant description parts of Embodiment I. The term "computer-readable storage medium" should be understood to include a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0083] Those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computer device. Optionally, they can be implemented by program codes executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. The present invention is not limited to any specific combination of hardware and software.

[0084] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0085] Although the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, it is not a limitation on the protection scope of the present invention. Those skilled in the art should understand that, based on the technical solutions of the present invention, various modifications or deformations that can be made by those skilled in the art without creative efforts are still within the protection scope of the present invention.

Claims

1. A method for detecting and identifying electromagnetic information leakage of indoor electronic equipment, characterized in that: include: Acquire multi-frequency signal data of indoor electronic equipment; A spectrum diagram of the signal is obtained based on the multi-frequency signal data, and the spectrum diagram is input into a pre-trained electromagnetic information leakage detection model for detection to obtain the device type to which the signal belongs; Input the multi-frequency signal data into a pre-trained dual-path feature fusion model for feature extraction to obtain the fingerprint feature of the signal; compare the fingerprint feature with the individual feature pre-stored in the feature library to obtain the device individual to which the signal belongs; The dual-path feature fusion model includes a first branch and a second branch in parallel, and a splicing layer in sequence. The first branch includes a transformation module, a first processing module, a second processing module, a third processing module, a fourth processing module and a fifth processing module in sequence. The second branch includes an embedding patch module, a position embedding module, a category embedding module, a normalization module, a multi-head attention module, a normalization module and a multi-layer perceptron in sequence.

2. The electromagnetic information leakage detection and identification method of indoor electronic equipment according to claim 1, characterized in that: Performing fast Fourier transform on the multi-frequency signal data to obtain a spectrum diagram of the signal.

3. The electromagnetic information leakage detection and identification method of indoor electronic equipment according to claim 1, characterized in that: The electromagnetic information leakage detection model includes a first convolution-batch normalization-nonlinear module, a second convolution-batch normalization-nonlinear module, a first bottleneck module, a second bottleneck module, a maximum pooling layer and a fully connected layer which are connected in sequence.

4. The electromagnetic information leakage detection and identification method of indoor electronic equipment according to claim 3, characterized in that: The first convolution-batch normalization-nonlinear module and the second convolution-batch normalization-nonlinear module have the same structure, including a 3×3 convolution layer, a batch normalization layer and an activation function layer connected in sequence.

5. The electromagnetic information leakage detection and identification method of indoor electronic equipment according to claim 3, characterized in that: The first bottleneck module and the second bottleneck module have the same structure, including a 1×1 convolution layer, an activation function layer, a 3×3 convolution layer, an activation function layer, a 1×1 convolution layer, a feature fusion layer and an activation function layer connected in sequence.

6. The electromagnetic information leakage detection and identification method of indoor electronic equipment according to claim 1, characterized in that: The first processing module includes a convolution-batch normalization-nonlinear module, a maximum pooling layer, a feature extraction module and an activation function layer in sequence; the second processing module, the third processing module, the fourth processing module and the fifth processing module have the same structure and include an instance normalization layer, a feature extraction module and an activation function layer in sequence.

7. The electromagnetic information leakage detection and identification method of indoor electronic equipment according to claim 6, characterized in that: The feature extraction module includes a first branch and a second branch, the first branch includes a 1×1 convolution-batch normalization-nonlinear module, a 3×3 convolution-batch normalization-nonlinear module and a 1×1 convolution-batch normalization-nonlinear module in sequence, and the second branch is a convolution layer.

8. An electromagnetic information leakage detection and identification system for indoor electronic equipment, characterized in that: include: A signal data acquisition module is configured to: acquire multi-frequency signal data of indoor electronic equipment; A device type detection module is configured to: obtain a spectrum diagram of the signal based on the multi-frequency signal data, input the spectrum diagram into a pre-trained electromagnetic information leakage detection model for detection, and obtain the device type to which the signal belongs; The target device identification module is configured to: input the multi-frequency signal data into a pre-trained dual-path feature fusion model for feature extraction to obtain the fingerprint feature of the signal; compare the fingerprint feature with the individual feature pre-stored in the feature library to obtain the device individual to which the signal belongs; The dual-path feature fusion model includes a first branch and a second branch in parallel, and a splicing layer in sequence. The first branch includes a transformation module, a first processing module, a second processing module, a third processing module, a fourth processing module and a fifth processing module in sequence. The second branch includes an embedding patch module, a position embedding module, a category embedding module, a normalization module, a multi-head attention module, a normalization module and a multi-layer perceptron in sequence.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the electromagnetic information leakage detection and identification method of an indoor electronic device as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the electromagnetic information leakage detection and identification method of an indoor electronic device as described in any one of claims 1-7 are implemented.

Images