An electromagnetic signal sorting and identification method fusing multi-dimensional and multi-scale features

Abstract

The present application relates to a kind of fusion multi-dimensional multi-scale feature electromagnetic signal sorting identification method, belong to electromagnetic signal processing field.The present application converts electromagnetic signal into constellation diagram, then converts constellation diagram into gray scale diagram;IQ data is converted into two-dimensional time-frequency diagram using time-frequency conversion mode;Using the image reduction method based on local mean, constellation diagram and time-frequency diagram are adjusted to uniform size;The time-frequency diagram and constellation diagram obtained are overlapped and arranged, obtain three-dimensional feature map with constellation diagram feature and different time-frequency feature simultaneously, and as the input of convolutional neural network.The present application improves the problem that signal sorting identification method accuracy is not high, and is greatly influenced by channel noise, artificial sorting method operation procedure is multiple, and is seriously dependent on professional personnel experience accumulation, automation degree is low and general-purpose expansion is insufficient.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of electromagnetic signal processing, and particularly relates to an electromagnetic signal sorting and identification method fusing multi-dimensional and multi-scale features. BACKGROUND

[0002] With the rapid development of modern communication technology and Internet of Things technology, the devices using electromagnetic spectrum and modulation modes gradually increase, making the electromagnetic environment increasingly complex, which brings many challenges to the development of electromagnetic signal sorting and identification technology.

[0003] Electromagnetic signal sorting and identification is a key technology to ensure subsequent demodulation processing at the receiving end, and plays an important role in both military and civilian fields. In the civilian field, it is mainly applied to electromagnetic spectrum monitoring and management, wireless signal monitoring and electromagnetic interference identification scenes; in the military field, it is mainly applied to electronic reconnaissance, electronic countermeasures, electronic intelligence and electromagnetic space security scenes, therefore, an effective method for sorting and identifying electromagnetic signals is of great significance.

[0004] The existing electromagnetic signal sorting and identification methods mainly include three categories: sorting and identification algorithm based on maximum likelihood theory, sorting and identification algorithm based on feature extraction and sorting and identification algorithm based on deep learning.

[0005] (1) The sorting and identification algorithm based on maximum likelihood theory makes a decision by comparing the received signal with the theoretically derived likelihood ratio with a threshold value. This method can obtain the optimal solution in theory, but has high computational complexity and needs a large amount of prior knowledge to support, which has limitations in the sorting and identification of non-cooperative signals in engineering applications.

[0006] (2) The sorting and identification algorithm based on feature extraction extracts signal features by artificial means and designs a classifier for sorting and identification. This method does not require a large amount of prior knowledge and reduces the amount of calculation, but it requires a high environment signal-to-noise ratio, and the generalization ability and robustness of the artificially extracted features are weak, making it difficult to achieve good adaptability and sorting and identification accuracy in the face of complex mode signals emerging constantly.

[0007] (3) The sorting and identification algorithm based on deep learning trains a deep network model by using electromagnetic big data to automatically extract deep signal features, overcoming the limitations of traditional artificial signal feature extraction sorting. For example, the use of conventional convolutional neural networks, improved capsule networks, convolutional neural networks and Choi-Williams distribution time-frequency diagrams, improved AlexNet networks and constellation diagrams can all achieve communication signal sorting and identification, but there are still technical problems such as low accuracy and insufficient universality and expansibility.

[0008] The application provides an electromagnetic signal sorting and identification method fusing multi-dimensional and multi-scale features, which first converts IQ data of electromagnetic signals into a two-dimensional constellation diagram and a time-frequency diagram; then performs size unification and normalization processing on the two-dimensional image, and then performs channel fusion on the optimized two-dimensional image, processes the fused image, obtains an electromagnetic signal two-dimensional image dataset, trains a multi-scale fusion convolutional neural network model using the dataset, and finally obtains a trained network model, which is used for sorting and identifying unknown signals.

[0009] The application can improve the technical problems of the prior art, such as low accuracy, great influence of channel noise, many operation procedures, serious dependence on professional experience accumulation, low automation degree and insufficient universality and expansibility. SUMMARY

[0010] (1) Technical problem to be solved

[0011] The application aims to provide an electromagnetic signal sorting and identification method fusing multi-dimensional and multi-scale features to solve the technical problems of the prior art, such as low accuracy, great influence of channel noise, many operation procedures, serious dependence on professional experience accumulation, low automation degree and insufficient universality and expansibility.

[0012] (2) Technical scheme

[0013] In order to solve the above technical problems, the application provides an electromagnetic signal sorting and identification method fusing multi-dimensional and multi-scale features, which comprises the following steps:

[0014] S1, converting electromagnetic signals into a two-dimensional constellation diagram: converting electromagnetic signals into a constellation diagram, and then converting the constellation diagram into a grayscale image;

[0015] S2, converting electromagnetic signals into a two-dimensional time-frequency diagram: converting IQ data into a two-dimensional time-frequency diagram by using a time-frequency conversion method;

[0016] S3, image optimization: adjusting the constellation diagram and the time-frequency diagram to a unified size by using a local mean-based image reduction method;

[0017] S4, multi-dimensional image fusion: overlapping and arranging the time-frequency diagram and the constellation diagram obtained in the above steps to obtain a three-dimensional feature diagram having both constellation diagram features and different time-frequency features, and taking the three-dimensional feature diagram as the input of a convolutional neural network;

[0018] S5, the multi-scale fusion convolutional neural network model establishment step: the structure of the multi-scale fusion convolutional neural network includes: double-branch series structure, multi-scale parallel branch structure, multi-branch residual structure and grouped convolution residual structure, the characteristics of electromagnetic signals are extracted and fused, and the final sorting prediction result is obtained.

[0019] (III) Beneficial effects

[0020] The application provides an electromagnetic signal sorting and identification method fusing multi-dimensional and multi-scale features, first converts IQ data of electromagnetic signals into a two-dimensional constellation diagram and a time-frequency diagram, then performs size unification and normalization processing on the two-dimensional image, then fuses channels of the optimized two-dimensional image, processes the fused image, obtains an electromagnetic signal two-dimensional image data set, trains a multi-scale fusion convolutional neural network model by using the data set, finally obtains a trained network model, and uses the trained model to sort and identify unknown signals.

[0021] The application can improve the technical problems of low accuracy of the existing conventional signal sorting and identification method, great influence of channel noise, many operation procedures of manual sorting method, serious dependence on professional experience accumulation, low automation degree and insufficient generalization and expansion. DETAILED DESCRIPTION

[0022] Figure 1 It is a method flowchart of the application;

[0023] Figure 2 It is a neural network structure diagram of the application;

[0024] Figure 3 It is a neural network module A structure diagram of the application;

[0025] Figure 4 It is a neural network module B1 structure diagram of the application;

[0026] Figure 5 It is a neural network module B2 structure diagram of the application;

[0027] Figure 6 It is a neural network module B3 structure diagram of the application;

[0028] Figure 7 It is a neural network module C1 structure diagram of the application;

[0029] Figure 8 It is a neural network module C2 structure diagram of the application;

[0030] Figure 9 It is a neural network module C3 structure diagram of the application;

[0031] Figure 10 Structure diagram of neural network module D1 of the present application;

[0032] Figure 11 Structure diagram of neural network module D2 of the present application;

[0033] Figure 12 Structure diagram of neural network module D3 of the present application. DETAILED DESCRIPTION

[0034] In order to make the purpose, content and advantages of the present application clearer, the specific embodiments of the present application are described in further detail below in combination with the drawings and examples.

[0035] The present application proposes an electromagnetic signal sorting and recognition method fusing multi-dimensional and multi-scale features. It includes the following steps:

[0036] (1) Step of converting electromagnetic signal into two-dimensional constellation, converting electromagnetic signal into constellation diagram, and then converting the constellation diagram into grayscale diagram.

[0037] (2) Step of converting electromagnetic signal into two-dimensional time-frequency diagram, adopting time-frequency conversion mode to convert IQ data into two-dimensional time-frequency diagram.

[0038] (3) Image optimization step, adopting image reduction method based on local mean to adjust the constellation diagram and time-frequency diagram to the same size.

[0039] (4) Multi-dimensional image fusion step, overlapping and arranging the time-frequency diagram and constellation diagram obtained in the above steps to obtain three-dimensional feature diagram with both constellation diagram features and different time-frequency features, and taking the three-dimensional feature diagram as the input of convolutional neural network.

[0040] (5) Step of establishing multi-scale fusion convolutional neural network model, the main structure of the multi-scale fusion convolutional neural network is double-branch series structure, multi-scale parallel branch structure, multi-branch residual structure and grouped convolution residual structure, which extracts and fuses the features of electromagnetic signal.

[0041] The present application converts electromagnetic signal into multiple two-dimensional images of different dimensions such as constellation diagram and time-frequency diagram and fuses them, automatically extracts multi-scale features of the signal and fuses them by using the trained convolutional neural network, so as to improve the technical problems of the existing traditional signal sorting and recognition method, such as low accuracy, great influence of channel noise, many operation procedures of manual sorting method, serious dependence on professional experience accumulation, low automation degree and insufficient generalization and expansion.

[0042] In view of the problems of low accuracy and low automation degree of existing electromagnetic signal sorting and identification methods, the application provides an electromagnetic signal sorting and identification method fusing multi-dimensional and multi-scale features, converts electromagnetic signals into two-dimensional images of multiple dimensions such as constellation diagrams and time-frequency diagrams, and fuses the images, and automatically extracts multi-scale features of the signals and performs fusion processing by using a trained convolutional neural network.

[0043] The application relates to an electromagnetic signal sorting and identification method fusing multi-dimensional and multi-scale features and a device thereof, and comprises the following steps.

[0044] S1, a step of converting electromagnetic signals into a two-dimensional constellation diagram, converting electromagnetic signals into a constellation diagram, and then converting the constellation diagram into a gray-scale diagram.

[0045] S2, a step of converting electromagnetic signals into a two-dimensional time-frequency diagram, and converting IQ data into a two-dimensional time-frequency diagram by using a time-frequency conversion method.

[0046] S3, an image optimization step, and the constellation diagram and the time-frequency diagram are adjusted to have a uniform size by using a local mean-based image reduction method.

[0047] S4, a multi-dimensional image fusion step, the time-frequency diagram and the constellation diagram obtained in the above steps are arranged in an overlapping manner to obtain a three-dimensional feature diagram having constellation diagram features and different time-frequency features at the same time, and the three-dimensional feature diagram is used as the input of a convolutional neural network.

[0048] S5, a multi-scale fusion convolutional neural network model establishment step, the multi-scale fusion convolutional neural network mainly has a double-branch series structure, a multi-scale parallel branch structure, a multi-branch residual structure and a grouped convolution residual structure, features of electromagnetic signals are extracted and fused to obtain a final sorting prediction result.

[0049] In order to realize the electromagnetic signal sorting and identification method fusing multi-dimensional and multi-scale features, the application provides a neural network device comprising four parts.

[0050] (1) a double-branch parallel structure, comprising one module, used for extracting shallow features of a feature map and adding a nonlinear excitation to improve the training speed and generalization ability of the model.

[0051] (2) a multi-scale parallel branch structure, comprising three modules, convolution kernels with different sizes are used for convolution operation to extract feature information of different scales of the image, or a maximum pooling operation is used to extract edge information and texture information of the image, and information fusion of multiple scale features is realized.

[0052] (3) a multi-branch residual structure, comprising three modules, residual convolution and batch normalization operations are performed on each module, nonlinear excitation is added, and the training speed and generalization ability of the model are improved.

[0053] (4) Grouped convolution residual structure. Including 3 modules. Each module carries out grouped residual reorganization and sorting identification.

[0054] Embodiment 1:

[0055] The present application will be described in detail below in conjunction with relevant drawings and specific implementation examples, but not as a limitation on the present application.

[0056] Figure 1 The flowchart of the method of the present application is shown as Figure 1 The present application relates to a multi-dimensional multi-scale feature fusion electromagnetic signal sorting identification method and device, which comprises the following steps:

[0057] (1) The electromagnetic signal is converted into a two-dimensional constellation diagram. The electromagnetic signal is converted into a constellation diagram, and then the constellation diagram is converted into a gray scale diagram.

[0058] In specific implementation, the in-phase component (I) and the quadrature component (Q) of the electromagnetic signal are mapped onto a two-dimensional plane to obtain a constellation diagram of the electromagnetic signal. Then, a new constellation diagram is generated according to the density of the scattered points of the constellation diagram to achieve feature enhancement of the constellation diagram. The gray scale conversion is performed on the feature-enhanced constellation diagram to obtain a gray scale diagram of the feature-enhanced constellation diagram.

[0059] (2) The electromagnetic signal is converted into a two-dimensional time-frequency diagram. The IQ data is converted into a two-dimensional time-frequency diagram by using a time-frequency transform method.

[0060] In specific implementation, the IQ data is converted into a two-dimensional time-frequency diagram by using a time-frequency transform method such as a linear time-frequency transform method like improved multi-synchro-squeezing transform (IMSST) and a nonlinear time-frequency transform method like smoothed pseudo Wigner-Ville distribution (SPWVD).

[0061] When the improved multi-synchro-squeezing transform is used for time-frequency transform, the signal model is assumed to be s(t)

[0062] Let where A(t) is the instantaneous frequency, and φ(t) is the instantaneous phase.

[0063] The second-order Taylor series expansion of s(t) is as follows:

[0064]

[0065] The short-time Fourier transform of s(t) is as follows:

[0066]

[0067] where p(t) is a window function, and let Then we have:

[0068]

[0069] Taking the partial derivative of S(t,ω) with respect to t, we have:

[0070]

[0071] According to the above formula, the instantaneous frequency estimation expression is

[0072]

[0073] The single synchro-squeezing transformation (SST) of S(t,ω) in the frequency domain can be obtained as:

[0074]

[0075] The expression after multiple synchro-squeezing transformations is:

[0076]

[0077] Taking the instantaneous frequency estimation and performing two rounding operations, we obtain the new frequency estimation expression:

[0078]

[0079]

[0080] where <·> is a rounding operator.

[0081] When the smoothed pseudo Wigner-Ville distribution is used for time-frequency analysis of a signal s(t), its expression is defined as:

[0082]

[0083] where z(t)=s(t)+jH[s(t)], z(t) is the analytic signal of s(t), and H[s(t)] is the Hilbert transform of s(t). g(u) is a frequency domain smoothing window function, and h(τ) is a time domain smoothing window function, and h(0)=G(0)=1.

[0084] (3) Image optimization step: the image reduction method based on local mean is used to adjust the constellation diagram and time-frequency diagram obtained in step two to the same size.

[0085] In specific implementation, the constellation diagram and the time-frequency diagram are respectively divided into matrix blocks according to a certain sampling interval, and then the mean value of the elements in each matrix block is calculated as the pixel corresponding to the adjusted image, and the pixel value is normalized. While retaining the characteristics of the original image as much as possible, the sizes of the constellation diagram and the time-frequency diagram are unified, and the size of the adjusted image is 227*227 pixels.

[0086] (4) A multi-dimensional image fusion step, the time-frequency diagram and the constellation diagram obtained in the above steps are arranged in an overlapping manner to obtain a three-dimensional feature diagram having both the constellation diagram characteristics and different time-frequency characteristics, and serving as the input of the convolutional neural network.

[0087] In specific implementation, in order to obtain features more discriminative than single-dimensional features, the two-dimensional images of the multiple-dimensional features of the IQ data transformation are fused to improve the distinguishability of different electromagnetic signals, and a three-dimensional feature diagram with a size of 227*227*3 is obtained, which is convenient for neural network processing.

[0088] (5) A multi-scale fusion convolutional neural network model establishment step, the multi-scale fusion convolutional neural network mainly has a double-branch series structure, a multi-scale parallel branch structure, a multi-branch residual structure, and a grouped convolution residual structure, which extracts and fuses the features of the electromagnetic signal.

[0089] To realize the electromagnetic signal sorting and identification method for fusing multi-dimensional and multi-scale features, the present application proposes a neural network model, which includes four parts:

[0090] (1) A double-branch parallel structure, including one module, which is used to extract the shallow features of the feature diagram and add a nonlinear excitation to improve the training speed and generalization ability of the model.

[0091] (2) A multi-scale parallel branch structure, including three modules. Different size convolution kernels are used for convolution operation to extract the feature information of different scales of the image, or a maximum pooling operation is used to extract the edge information and texture information of the image, and the information fusion of multiple scale features is realized.

[0092] (3) A multi-branch residual structure, including three modules. Each module performs residual convolution, batch normalization operation, and nonlinear excitation to improve the training speed and generalization ability of the model.

[0093] (4) A grouped convolution residual structure, including three modules. Each module performs grouped residual reorganization and sorting and identification.

[0094] The part (1) is a double-branch parallel structure, including one module A, which has a structure as shown in Figure 3 .

[0095] The main function of module A is to extract the shallow features of the feature map. The asymmetric convolution of 1x7 and 7x1 is used to increase the diversity of the extracted features, while reducing the computational complexity. After convolution, each branch of the module is subjected to batch normalization (BN) operation and added with a nonlinear excitation to improve the training speed and generalization ability of the model.

[0096] The dimension of the input feature map of the neural network is 227x227x3. The input feature map is output to module A as a feature map with a size of 111x111x64 after 3 convolution layers with a size of 3x3. The double-branch parallel structure of module A is composed of different types of convolution. The upper layer feature map is input to the double-branch parallel structure composed of 3x3 max pooling layer and 3x3 convolution layer. After channel splicing, it is output to the double-branch parallel structure composed of 1x1 convolution layer, 3x3 convolution layer and 1x1 convolution layer, 1x7 convolution layer, 7x1 convolution layer and 3x3 convolution layer. After channel splicing, it is output to the double-branch parallel structure composed of 3x3 max pooling layer and 3x3 convolution layer. After channel splicing again, a feature map with a size of 26x26x384 is output to module B1.

[0097] The part (2) is a multi-scale parallel branch structure. It includes three modules B1, B2 and B3, the structures of which are shown in Figure 4 , Figure 5 , Figure 6

[0098] ​The modules B1, B2 and B3 are multi-scale parallel branch structures, each of which is composed of two or more branch structures, each of which uses a convolution kernel of different size to perform convolution operation to extract feature information of different scales of the image, or uses a max-pooling operation to extract edge information and texture information of the image, and then stacks the feature maps of the same size output by the four branches to realize information fusion of multiple scale features. The module B1 has four branches, the first branch is a 1*1 convolution, the second branch is a 1*1 convolution connected in series with a 5*5 convolution, the third branch is a 1*1 convolution connected in series with two 3*3 convolutions, and the fourth branch is a 3*3 max-pooling connected in series with a 1*1 convolution. The four branches are connected in series to output a feature map with a size of 26*26*320; the module B2 has three branches, the first branch is a 3*3 convolution, the second branch is a 1*1 convolution connected in series with two 3*3 convolutions, and the third branch is a 3*3 max-pooling. The three branches are connected in series to output a feature map with a size of 12*12*1088; the module B3 has four branches, the first branch is a 1*1 convolution connected in series with a 3*3 convolution, the second branch is a 1*1 convolution connected in series with a 3*3 convolution, the third branch is a 1*1 convolution connected in series with two 3*3 convolutions, and the fourth branch is a 3*3 max-pooling. The four branches are connected in series to output a feature map with a size of 5*5*2080. The multi-scale parallel branch structure is connected to the grouped convolution residual structure after channel splicing, the module B1 is connected to the module D1, then the module D1 is connected to the module C1, the module B2 is connected to the module D2, then the module D2 is connected to the module C2, and the module B3 is connected to the module D3, then the module D3 is connected to the module C3. After the convolution operation of each module branch, a batch normalization operation is performed, and a nonlinear excitation is added to improve the training speed and generalization ability of the model.

[0099] The part (3) is a multi-branch residual structure, which includes three modules C1, C2 and C3, the structures of which are shown in Figure 7 、 Figure 8 、 Figure 9

[0100] ​Module C1, C2, C3 is a multi-branch residual structure, which is mainly composed of more than two branch structures. Module C1 has four branches. Branch one is a 1x1 convolution. Branch two is a 1x1 convolution connected in series with a 3x3 convolution. Branch three is a 1x1 convolution connected in series with two 3x3 convolutions. After channel splicing, it is connected to a 1x1 convolution and the residual is reduced. After adding the residual of the cross-layer connection line of branch four, a Relu function is added as a nonlinear excitation, and a 26x26x320 feature map is output to module D1. Module C2 has three branches. Branch one is a 1x1 convolution. Branch two is a 1x1 convolution connected in series with a 1x7 convolution layer and a 7x1 convolution layer. After channel splicing, it is connected to a 1x1 convolution and the residual is reduced. After adding the residual of the cross-layer connection line of branch three, a Relu function is added as a nonlinear excitation, and a 12x12x1088 feature map is output to module D2. Module C3 has three branches. Branch one is a 1x1 convolution. Branch two is a 1x1 convolution connected in series with a 1x3 convolution layer and a 3x1 convolution layer. After channel splicing, it is connected to a 1x1 convolution and the residual is reduced. After adding the residual of the cross-layer connection line of branch three, a Relu function is added as a nonlinear excitation, and a 5x5x2080 feature map is output to module D3. There are six modules C1, twelve modules C2 and six modules C3 in the entire network model. After the convolution operation of each module branch, batch normalization operation is performed, and nonlinear excitation is added to improve the training speed and generalization ability of the model.

[0101] The part (4) is a grouped convolution residual structure. It includes three modules D1, D2 and D3, the structures of which are shown in Figure 10 、 Figure 11 、 Figure 12

[0102] Module D1, D2, D3 is a grouped convolution residual structure, which is mainly composed of cross-layer connection lines and grouped convolution. The output of the upper layer is subjected to 1x1 point-by-point grouped convolution, batch normalization operation, and nonlinear excitation is added after channel reorganization. The channel reorganization diagram is shown in Figure 12 ​The feature map size of the upper layer output is assumed to be w x h x d, the feature map is divided into g groups, each group has n feature maps, and the dimension of the grouped feature map can be represented as w x h x g x n. The rearrangement and transposition are performed along the g axis and the n axis direction, then the transposed feature map is reorganized to obtain a feature map with a dimension of w x h x d, the information exchange between different groups is realized, then a 3 x 3 depth separable convolution is connected, and a batch normalization operation is performed, and then the 1 x 1 point grouping convolution is added to the cross-layer connection line, and then output to the next layer. The point grouping convolution in branch one of module D1 is connected to the 3 x 3 depth separable convolution after channel reorganization, the parameter g of the depth separable convolution is 320, and then connected to the point grouping convolution and batch normalization operation, and the residual of the cross-layer connection line of branch two is spliced through the channel, and the feature map with a size of 26 x 26 x 320 is output to the next layer. The point grouping convolution in branch one of module D2 is connected to the 3 x 3 depth separable convolution after channel reorganization, the parameter g of the depth separable convolution is 1088, and then connected to the point grouping convolution and batch normalization operation, and the residual of the cross-layer connection line of branch two is spliced through the channel, and the feature map with a size of 12 x 12 x 1088 is output to the next layer; the point grouping convolution in branch one of module D3 is connected to the 3 x 3 depth separable convolution after channel reorganization, the parameter g of the depth separable convolution is 2080, and then connected to the point grouping convolution and batch normalization operation, and the residual of the cross-layer connection line of branch two is spliced through the channel, and the feature map with a size of 5 x 5 x 2080 is output to the next layer. The structures of modules D2 and D3 are basically the same as those of D1, the main difference is that the number of groups is different, the size of the output feature map of module D2 is 12 x 12 x 1088, and the size of the output feature map of module D3 is 5 x 5 x 2080. After channel splicing, the feature map with a size of 5 x 5 x 2080 is output to the 1 x 1 convolution, global average pooling, full connection layer, and the final sorting prediction result is obtained by using the Softmax classifier for classification.

[0103] The application converts electromagnetic signals into two-dimensional images of multiple dimensions such as constellation diagrams and time-frequency diagrams, and fuses them. The trained convolutional neural network automatically extracts multi-scale features of the signals and fuses them to improve the technical problems of the existing traditional signal sorting and recognition method, such as low accuracy, great influence of channel noise, multiple operation procedures, serious dependence on professional experience accumulation, low automation degree, and insufficient universality and expansibility.

[0104] The above only describes the preferred embodiments of the application, and it should be noted that those skilled in the art can make several improvements and modifications without departing from the technical principles of the application, and these improvements and modifications should also be considered within the protection scope of the application.

Claims

1. A method for sorting and identifying electromagnetic signals by fusing multi-dimensional multi-scale features, characterized in that, The method comprises the following steps: S1, electromagnetic signal conversion to two-dimensional constellation diagram step: convert the electromagnetic signal into a constellation diagram, and then convert the constellation diagram into a grayscale image; S2, electromagnetic signal conversion to two-dimensional time-frequency diagram step: convert the IQ data into a two-dimensional time-frequency diagram by using a time-frequency conversion method; S3, image optimization step: adjust the constellation diagram and the time-frequency diagram to a unified size by using a local mean-based image reduction method; S4, multi-dimensional image fusion step: overlap the time-frequency diagram and the constellation diagram obtained in the above steps to obtain a three-dimensional feature diagram having both constellation diagram features and different time-frequency features, and use the three-dimensional feature diagram as the input of a multi-scale fusion convolutional neural network model; S5, multi-scale fusion convolutional neural network model establishment step: the structure of the multi-scale fusion convolutional neural network comprises a double-branch parallel structure, a multi-scale parallel branch structure, a multi-branch residual structure, and a grouped convolution residual structure, extracts features of the electromagnetic signal, and fuses the features to obtain a final sorting prediction result; In the step S5, the double-branch parallel structure comprises one module A for extracting shallow features of the feature diagram and adding a nonlinear excitation to improve the training speed and generalization ability of the model; the multi-scale parallel branch structure comprises three modules B1, B2, and B3 for performing convolution operations using convolution kernels of different sizes to extract feature information of different scales of the image, or using a maximum pooling operation to extract edge information and texture information of the image, and realizing information fusion of multiple scale features; the multi-branch residual structure comprises three modules C1, C2, and C3 for performing residual convolution and batch normalization operations, and nonlinear excitation to improve the training speed and generalization ability of the model; the grouped convolution residual structure comprises three modules D1, D2, and D3 for performing grouped residual reorganization and sorting identification. In the step S1, the in-phase component I and the quadrature component Q of the electromagnetic signal are mapped onto a two-dimensional plane to obtain a constellation diagram of the electromagnetic signal; a new constellation diagram is generated according to the density of the scatter points of the constellation diagram to realize feature enhancement of the constellation diagram, and a grayscale conversion is performed on the feature-enhanced constellation diagram to obtain a grayscale image of the feature-enhanced constellation diagram.

2. The method of claim 1, wherein the fusion multi-dimensional multi-scale feature of the electromagnetic signal is sorted and identified. In the step S2, the time-frequency conversion method is an improved multiple synchronous compression transform (IMSST) method and a smooth pseudo Wigner-Ville distribution (SPWVD) transform method. 3.The method of claim 1, wherein, The step S3 specifically comprises: dividing the constellation diagram and the time-frequency diagram into matrix blocks according to a certain sampling interval, then calculating the mean value of the elements in each matrix block as the corresponding pixel of the adjusted image, and performing normalization processing on the pixel value to realize the size unification of the constellation diagram and the time-frequency diagram. 4.The method of claim 1, wherein, The unified size is 227x227 pixels.

5. The method of claim 4, wherein the fusion multi-dimensional multi-scale feature of the electromagnetic signal is sorted and identified. ​ 6. The method of claim 1, wherein the fusion multi-dimensional multi-scale feature of the electromagnetic signal is sorted and identified. The double-branch parallel structure comprises one module A; the dimension of the neural network input feature map is 227*227*3, the input feature map is output to the module A as a feature map with a size of 111*111*64 through three 3*3 convolution layers; each branch of the double-branch parallel structure of the module A is composed of different types of convolution, the upper layer feature map is input to the double-branch parallel structure composed of a 3*3 max-pooling layer and a 3*3 convolution layer, is output to the double-branch parallel structure composed of a 1*1 convolution layer, a 3*3 convolution layer and a 1*1 convolution layer, a 1*7 convolution layer, a 7*1 convolution layer and a 3*3 convolution layer after channel splicing, is output to the double-branch parallel structure composed of a 3*3 max-pooling layer and a 3*3 convolution layer after channel splicing again, and is output to the module B1 as a feature map with a size of 26*26*384 after channel splicing again.

7. The method of claim 6, wherein the fusion multi-dimensional multi-scale feature of the electromagnetic signal is sorted and identified. The multi-scale parallel branch structure comprises three modules B1, B2 and B3; the module B1 has four branches, the first branch is a 1*1 convolution, the second branch is a 1*1 convolution and a 5*5 convolution in series, the third branch is a 1*1 convolution and two 3*3 convolutions in series, and the fourth branch is a 3*3 max-pooling and a 1*1 convolution in series, and the four branches are output as a feature map with a size of 26*26*320 after channel splicing; the module B2 has three branches, the first branch is a 3*3 convolution, the second branch is a 1*1 convolution and two 3*3 convolutions in series, and the third branch is a 3*3 max-pooling, and the three branches are output as a feature map with a size of 12*12*1088 after channel splicing; the module B3 has four branches, the first branch is a 1*1 convolution and a 3*3 convolution in series, the second branch is a 1*1 convolution and a 3*3 convolution in series, the third branch is a 1*1 convolution and two 3*3 convolutions in series, and the fourth branch is a 3*3 max-pooling, and the four branches are output as a feature map with a size of 5*5*2080 after channel splicing; the multi-scale parallel branch structure is connected to the grouped convolution residual structure after channel splicing, the module B1 is connected to the module D1, then the module D1 is connected to the module C1, the module B2 is connected to the module D2, then the module D2 is connected to the module C2, the module B3 is connected to the module D3, and then the module D3 is connected to the module C3; the convolution operation of each module branch is followed by a batch normalization operation, and a nonlinear excitation is added.

8. The method of claim 7, wherein the fusion multi-dimensional multi-scale feature of the electromagnetic signal is sorted and identified. The multi-branch residual structure comprises three modules C1, C2 and C3; the module C1 has four branches, the first branch is a 1*1 convolution, the second branch is a 1*1 convolution connected with a 3*3 convolution in series, the third branch is a 1*1 convolution connected with two 3*3 convolutions in series, after channel splicing, it is connected to a 1*1 convolution and the residual is reduced, after the residual of the fourth branch is added to the cross-layer connection line, a Relu function is used as a nonlinear excitation, and a feature map with a size of 26*26*320 is output to the module D1; the module C2 has three branches, the first branch is a 1*1 convolution, the second branch is a 1*1 convolution connected with a 1*7 convolution layer and a 7*1 convolution layer in series, after channel splicing, it is connected to a 1*1 convolution and the residual is reduced, after the residual of the third branch is added to the cross-layer connection line, a Relu function is used as a nonlinear excitation, and a feature map with a size of 12*12*1088 is output to the module D2; the module C3 has three branches, the first branch is a 1*1 convolution, the second branch is a 1*1 convolution connected with a 1*3 convolution layer and a 3*1 convolution layer in series, after channel splicing, it is connected to a 1*1 convolution and the residual is reduced, after the residual of the third branch is added to the cross-layer connection line, a Relu function is used as a nonlinear excitation, and a feature map with a size of 5*5*2080 is output to the module D3; there are six modules C1, twelve modules C2 and six modules C3 in the entire network model, and batch normalization operation is performed after the convolution operation of each module branch, and a nonlinear excitation is added.

9. The method of claim 8, wherein the fusion multi-dimensional multi-scale feature of the electromagnetic signal is sorted and identified. The grouped convolution residual structure comprises three modules D1, D2 and D3; the first branch of the module D1 is point-by-point grouped convolution, which is connected to a 3*3 depth separable convolution after channel reorganization, then connected to point-by-point grouped convolution and batch normalization operation, and the residual of the second branch of the cross-layer connection line is channel spliced, and a feature map with a size of 26*26*320 is output to the next layer; the first branch of the module D2 is point-by-point grouped convolution, which is connected to a 3*3 depth separable convolution after channel reorganization, and the parameter g of the depth separable convolution is 1088, then connected to point-by-point grouped convolution and batch normalization operation, and the residual of the second branch of the cross-layer connection line is channel spliced, and a feature map with a size of 12*12*1088 is output to the next layer; the first branch of the module D3 is point-by-point grouped convolution, which is connected to a 3*3 depth separable convolution after channel reorganization, and the parameter g of the depth separable convolution is 2080, then connected to point-by-point grouped convolution and batch normalization operation, and the residual of the second branch of the cross-layer connection line is channel spliced, and a feature map with a size of 5*5*2080 is output to the next layer; after channel splicing, the last module D3 outputs a feature map with a size of 5*5*2080 to a 1*1 convolution, performs global average pooling and full connection layer, and uses a Softmax classifier to classify to obtain the final sorting prediction result.

Images