Data enhancement electromagnetic interference identification method based on MobilenetV4

Abstract

The invention discloses a data enhancement electromagnetic interference identification method based on MobilenetV4, belongs to the field of electromagnetic protection of an engine electronic control system, and aims to perform data enhancement of a data set by using a deep convolutional generative adversarial network and perform electromagnetic interference identification by using an improved DeepLabV3 + network taking the MobilenetV4 as a backbone network. In the field of electromagnetism, a data set is difficult to manufacture, and the sample size is generally small. According to the method, the deep convolutional generative adversarial network is used for expanding the number of data sets, and then the expanded data sets are input into the improved DeepLabV3 + network to identify electromagnetic interference, so that normal signals and interference signals are correctly distinguished.

Description

Technical Field

[0001] The present invention relates to the field of electromagnetic protection of engine electronic control systems, and in particular to a data enhanced electromagnetic interference recognition method based on MobilenetV4. Background Art

[0002] Electromagnetic interference (EMI) refers to the adverse effects of electromagnetic fields on the normal operation of electronic equipment. Electromagnetic interference can be generated by mutual interference between internal components of electronic equipment, magnetic fields generated by high-power equipment, coupling of electromagnetic energy, etc. Strong electromagnetic interference may cause damage to electronic equipment, affect data transmission and the normal operation of the system. Common electromagnetic interference hardware protection methods include grounding, shielding and filtering. However, grounding technology requires additional wiring and grounding equipment, which increases cost and complexity; additional shielding materials increase cost and weight, and different equipment requires different shielding methods, which has poor applicability; filters may cause signal distortion, and high-performance filters are expensive. Therefore, traditional hardware protection methods cannot meet the current requirements of electromagnetic protection due to their high cost, complex structure and poor applicability. Therefore, the industry needs an intelligent, low-cost, fast and accurate electromagnetic signal detection method to identify electromagnetic interference. Summary of the invention

[0003] The present invention provides a data-enhanced electromagnetic interference recognition method based on MobilenetV4, which uses a deep convolutional generative adversarial network (DCGAN) to expand the number of data sets, and then inputs the expanded data sets into an improved DeepLabV3+ network to identify electromagnetic interference, thereby correctly distinguishing normal signals from interference signals.

[0004] The embodiment of the present invention provides a data enhanced electromagnetic interference identification method based on MobilenetV4, comprising the following steps:

[0005] Step 1: Use a speed sensor to collect the speed signal to obtain a one-dimensional signal sample containing normal and electromagnetic interference, perform short-time Fourier transform on the one-dimensional signal sample to generate a two-dimensional time-frequency diagram, and obtain a speed signal sample data set;

[0006] Step 2: Use the DCGAN enhanced data set to expand the number of the speed signal sample data set, and divide the expanded speed signal sample data set into a training data set and a test data set in proportion;

[0007] Step 3: Integrate MobileNetV4 into the DeepLabV3+ architecture as the encoder of the semantic segmentation model;

[0008] Step 4: Using the training data set to train the semantic segmentation model, and using the test data set to test the training effect of the semantic segmentation model;

[0009] Step 5: Use the trained semantic segmentation model to identify the two-dimensional time-frequency diagram of the signal to be identified, and output the semantic segmentation result to identify the normal speed signal and interference signal in the two-dimensional time-frequency diagram.

[0010] Optionally, in one embodiment of the present invention, in step 1, after performing short-time Fourier transform on the one-dimensional signal sample to generate a two-dimensional time-frequency diagram, the two-dimensional time-frequency diagram is converted into a JPG format, and a global threshold binarization method is used to generate the speed signal sample data set.

[0011] Optionally, in one embodiment of the present invention, in step 1, a label is assigned to each sample in the speed signal sample data set, and the categories of images in the speed signal sample data set include three categories, the first category is background; the second category is normal speed signal; the third category is interference signal from electromagnetic pulse; use the open source image annotation tool labelme to complete the annotation of the data set, select the folder of the image to be annotated after entering the software, and open the image to be annotated; before starting the annotation, define the categories that need to be marked in the image; select polygon annotation, and click on each vertex of the polygon to be drawn on the image in turn; after completing the annotation, save the annotation result of the current image, and process the output in JSON format and convert it into MASK format.

[0012] Optionally, in one embodiment of the present invention, in step 2, DCGAN is a generative adversarial network that combines deep learning and convolutional neural networks, consisting of a generator and a discriminator, and the structure of the generator is as follows: the input layer receives a random noise vector as input; the fully connected layer maps the input noise vector into a higher dimensional feature vector; the transposed convolution layer gradually increases the spatial dimension of the feature map to restore the spatial resolution of the image; the batch normalization layer is used after each convolution layer to speed up the training speed and stabilize the learning process of the model; the last layer uses tanh as the activation function to limit the range of the output value to [-1, 1], and other layers use ReLU as the activation function; the output layer outputs a generated image with the same size and number of channels as the training data.

[0013] Optionally, in one embodiment of the present invention, a DCGAN network discriminator is used to distinguish whether an input image is a real image from training data or a fake image generated by a generator, and the structure is as follows: a convolution layer extracts features of the input image, and the convolution step is set to 2; a batch normalization layer is used in layers other than the last layer; the LeakyReLU function is selected as the activation function; a fully connected layer is used to further process features and perform classification, and is used after the convolution layer; the output layer is a fully connected layer with a sigmoid activation function, and the range of the output value is limited to [0,1], indicating the probability that the input image is a real image.

[0014] Optionally, in one embodiment of the present invention, in step 3, MobileNetV4 is used to replace the original Xception network to enhance the encoder feature extraction capability. The original structure of the model framework is the semantic segmentation architecture of DeepLabV3+, and an encoder-decoder structure is adopted. In the encoder part, an improved Xception backbone network and a spatial pyramid pooling module are included. The depth-separable convolution operations of different channels in the Xception network model are used to segment multiple image features. The spatial pyramid pooling module is adopted at the end of the encoder to identify the multi-scale semantic features of the image. The decoder part successively performs 4-fold bilinear upsampling, feature fusion, 3*3 convolution and 4-fold bilinear upsampling operations to complete the refinement of the boundaries of each target feature of the image and obtain the final feature segmentation result.

[0015] Optionally, in one embodiment of the present invention, in step 5, the semantic segmentation model assigns each pixel in the two-dimensional time-frequency map to a specific category to distinguish different objects and boundaries in the image, inputs the two-dimensional time-frequency map, and outputs a pixel-level classification map of the same size as the input image, wherein each pixel corresponds to a category label, and the category labels include background, normal rotation speed signal, and interference signal.

[0016] The data enhancement electromagnetic interference identification method based on MobilenetV4 of the embodiment of the present invention uses a deep convolutional generative adversarial network to perform data enhancement of the data set and an improved DeepLabV3+ network with MobilenetV4 as the backbone network to perform electromagnetic interference identification. In the electromagnetic field, the production of data sets is difficult and the sample size is usually small. The present invention uses a deep convolutional generative adversarial network to expand the number of data sets, and then inputs the expanded data set into the improved DeepLabV3+ network to identify electromagnetic interference, thereby correctly distinguishing normal signals from interference signals.

[0017] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A flowchart of a data enhanced electromagnetic interference identification method based on MobilenetV4 according to an embodiment of the present invention;

[0020] Figure 2 It is a schematic diagram of the execution process of the data enhanced electromagnetic interference identification method based on MobilenetV4 in an embodiment of the present invention;

[0021] Figure 3 This is a network structure diagram of the data enhanced electromagnetic interference identification method based on MobilenetV4 in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0023] like Figure 1 and Figure 2 As shown, the data-enhanced electromagnetic interference identification method based on MobilenetV4 includes the following steps:

[0024] Step 1: Use a speed sensor to collect the speed signal to obtain a one-dimensional sample of the signal containing normal and electromagnetic interference. Perform short-time Fourier transform on the one-dimensional sample of the signal to generate a two-dimensional time-frequency diagram to obtain a speed signal sample data set.

[0025] When using short-time Fourier transform to generate a two-dimensional time-frequency diagram, the electromagnetic interference signal is a non-stationary signal. A single time domain or frequency domain analysis method cannot accurately obtain the signal information. The short-time Fourier transform can display the frequency components of the signal at different time points, which is conducive to signal identification.

[0026] Specifically, the time-frequency graph is converted into a JPG format image to meet the format requirements of deep learning for data. In order to improve the image quality and highlight the features of the target area in the time-frequency graph, a global threshold binarization method is used to generate a training data set.

[0027] The short-time Fourier transform uses a short-time window function to divide the entire non-stationary signal into stationary signals of several time periods, and then performs Fourier transform on the divided signals. The sliding short-time window function can obtain the frequency distribution of the signal over the entire time. The formula is as follows:

[0028]

[0029] Here, h(·) represents the short-time window function, τ represents the time shift, and ω=2πf represents the frequency information of the x signal.

[0030] The global threshold method selects a fixed global threshold. All pixels above this threshold are set to white, and pixels below the threshold are set to black. The formula is as follows:

[0031]

[0032] Where f(x,y) is the pixel value of point (x,y), g(x,y) is the segmented image, and T is the global threshold, which is usually obtained through a histogram.

[0033] In step 1, each sample in the data set is assigned a label, which is used for subsequent training and evaluation of the model. There are three types of image categories: the first is background; the second is normal speed signal; and the third is interference signal from electromagnetic pulse. Use the open source image annotation tool labelme to complete the annotation of the data set. After entering the software, select the folder of the image to be annotated and open the image to be annotated; before starting the annotation, define the categories that need to be marked in the image; select polygon annotation, and click on each vertex of the polygon to be drawn on the image in turn; after completing the annotation, save the annotation result of the current image, and process the output in JSON format and convert it to MASK format.

[0034] Step 2: Use the DCGAN enhanced data set to expand the number of the speed signal sample data set, and divide the expanded speed signal sample data set into a training data set and a test data set in proportion.

[0035] In a specific embodiment, the samples are divided into a training data set and a test data set in a data ratio of 6:4, and it is ensured that all samples in the test data set are real samples.

[0036] In step 2, DCGAN is a generative adversarial network that combines deep learning and convolutional neural networks. The generator generates realistic images as much as possible. It consists of a generator and a discriminator. The structure of the generator is as follows: the input layer receives a random noise vector as input; the fully connected layer maps the input noise vector into a higher-dimensional feature vector; the transposed convolution layer gradually increases the spatial dimension of the feature map to restore the spatial resolution of the image; the batch normalization layer is used after each convolution layer to speed up the training and stabilize the learning process of the model; the last layer uses tanh as the activation function to limit the range of the output value to [-1, 1], and other layers use ReLU as the activation function; the output layer outputs a generated image with the same size and number of channels as the training data.

[0037] The DCGAN network discriminator is used to distinguish whether the input image is a real image from the training data or a fake image generated by the generator. The structure is as follows: the convolution layer extracts the features of the input image, and the convolution step is set to 2 to gradually reduce the spatial dimension of the image; the batch normalization layer is used in all layers except the last layer to accelerate training and improve model stability; the LeakyReLU function is selected as the activation function to allow negative gradients to have a certain transmission; the fully connected layer is used to further process the features and perform classification, and is used after the convolution layer; the output layer is a fully connected layer with a sigmoid activation function, and the range of the output value is limited to [0,1], indicating the probability that the input image is a real image.

[0038] Specifically, DCGAN is a generative adversarial network that combines deep learning and convolutional neural networks. Its main structure is the generator and the discriminator. The function of the generator is to synthesize data similar to the real data set. Its formula is as follows:

[0039] G * =argminDiv(P data(x) ,P G )

[0040] Among them, P data(x) and P G They represent the distribution of generated sample data and real sample data respectively, and Div refers to the distance between the two distributions.

[0041] The purpose of the discriminator is to distinguish whether the input image is a real image from the training dataset or a fake image generated by the generator. The formula is as follows:

[0042] D * = argmaxDiv(P G ,P data )

[0043] In the process of training the discriminant model, the input sample of the generator model is assigned label 0, and the real input sample is assigned label 1. The objective function value of the discriminator represents the true probability of the input sample, and its formula is as follows:

[0044]

[0045] in, and are the mathematical expectations of the real data x and random noise data z respectively; D(x) is the output of the real data x through the discriminator, and G(z) is the output of the random noise z through the generator.

[0046] Step 3: Integrate MobileNetV4 into the DeepLabV3+ architecture as the encoder of the semantic segmentation model.

[0047] In step 3, MobileNetV4 is used to replace the original Xception network to enhance the encoder feature extraction capability. The original structure of the model framework is the semantic segmentation architecture of DeepLabV3+, which adopts an encoder-decoder structure. In the encoder part, it includes an improved Xception backbone network and a spatial pyramid pooling module (ASPP). The depth-separable convolution operations of different channels in the Xception network model are used to segment multiple image features. The spatial pyramid pooling module is used at the end of the encoder to identify the multi-scale semantic features of the image. The decoder part performs 4x bilinear upsampling, feature fusion, 3*3 convolution and 4x bilinear upsampling operations in succession to refine the boundaries of each target feature of the image and obtain the final feature segmentation result.

[0048] MobileNetV4 is integrated into the DeepLabV3+ architecture as the encoder backbone network. Specifically, a universal inverted bottleneck block is introduced into the encoder. The universal inverted bottleneck block contains an optional deep convolutional layer for expanding and projecting feature maps. The improved backbone network integrates efficient multi-scale feature extraction functions. Compared with the traditional architecture, it reduces model parameters while maintaining the decoder accuracy and thus reducing the computational load. Figure 3 shown.

[0049] Step 4: Use the training data set to train the semantic segmentation model, and use the test data set to test the training effect of the semantic segmentation model.

[0050] The training data set includes annotated images of pictures. For each training image, an annotated image is provided in which the value of each pixel indicates the category to which the pixel belongs. During the training process, the model will perform forward propagation based on the training image, generate a prediction for each pixel, and compare it with the true label in the annotated image to calculate the error and perform back propagation, thereby continuously optimizing the model parameters and ultimately enabling the model to accurately perform segmentation.

[0051] In step 4, the semantic segmentation model is trained and the parameters are optimized. Specifically, the computing platform uses CPU AMD Ryzen 9 7900X 12-Core Processor and GPU NVIDIA RTX 3060, the deep learning framework is Pytorch, the training parameter batch size is 4, the optimizer is Lion, the learning rate is 0.0001, and the weight decay (L2 regularization) is 0.00001. The learning rate scheduling algorithm uses a combination of StepLR scheduler and ReduceLROnPlateau scheduler. The former attenuates the learning rate according to a fixed factor after each epoch, and the latter reduces the learning rate when the performance does not improve.

[0052] Step 5: Use the trained semantic segmentation model to identify the two-dimensional time-frequency diagram of the signal to be identified, and output the semantic segmentation result to identify the normal speed signal and interference signal in the two-dimensional time-frequency diagram.

[0053] In step 5, the semantic segmentation model assigns each pixel in the two-dimensional time-frequency map to a specific category to distinguish different objects and boundaries in the image. The two-dimensional time-frequency map is input and a pixel-level classification map of the same size as the input image is output, in which each pixel corresponds to a category label, including background, normal speed signal and interference signal.

[0054] According to the data enhancement electromagnetic interference recognition method based on MobilenetV4 proposed in the embodiment of the present invention, it is based on the improved DeepLabV3+ network, uses the DCGAN network to expand the data set, solves the pain point of the small number of samples, and uses MobilenetV4 to replace the backbone network of DeepLabV3+. MobilenetV4 uses deep separable convolution to decompose the standard convolution operation into deep convolution and point-by-point convolution, which can reduce the number of parameters and the amount of calculation, and has high computational efficiency. The hollow convolution is used to expand the receptive field of the convolution operation, better capture the edges and details of the target, and the decoder module helps to improve the spatial resolution of the segmentation result; the multi-scale input fusion technology can process targets at different scales and complete the final electromagnetic interference recognition task. When the computing platform uses the CPU AMD Ryzen 97900X 12-Core Processor and the GPU NVIDIA RTX 3060, the average category accuracy of this recognition method is about 94%, and the FPS can reach 78. The model has the characteristics of fast computing speed, short training time, low memory usage, and high accuracy.

[0055] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0056] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0057] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

Claims

1. A data-enhanced electromagnetic interference identification method based on MobilenetV4, characterized in that: The following steps are involved: Step 1: Use a speed sensor to collect the speed signal to obtain a one-dimensional signal sample containing normal and electromagnetic interference, perform short-time Fourier transform on the one-dimensional signal sample to generate a two-dimensional time-frequency diagram, and obtain a speed signal sample data set; Step 2: Use the DCGAN enhanced data set to expand the number of the speed signal sample data set, and divide the expanded speed signal sample data set into a training data set and a test data set in proportion; Step 3: Integrate MobileNetV4 into the DeepLabV3+ architecture as the encoder of the semantic segmentation model; Step 4: Using the training data set to train the semantic segmentation model, and using the test data set to test the training effect of the semantic segmentation model; Step 5: Use the trained semantic segmentation model to identify the two-dimensional time-frequency diagram of the signal to be identified, and output the semantic segmentation result to identify the normal speed signal and interference signal in the two-dimensional time-frequency diagram.

2. The method according to claim 1, characterized in that: In step 1, after performing short-time Fourier transform on the one-dimensional signal sample to generate a two-dimensional time-frequency diagram, the two-dimensional time-frequency diagram is converted into a JPG format, and a global threshold binarization method is used to generate the speed signal sample data set.

3. The method according to claim 1, characterized in that In step 1, a label is assigned to each sample in the speed signal sample data set. The categories of images in the speed signal sample data set include three categories: the first category is background; the second category is normal speed signal; the third category is interference signal from electromagnetic pulse; the open source image annotation tool labelme is used to complete the annotation of the data set. After entering the software, select the folder of the image to be annotated and open the image to be annotated; before starting the annotation, define the category to be marked in the image; select polygon annotation, and click on each vertex of the polygon to be drawn on the image in turn; After the annotation is completed, the annotation results of the current image are saved, and the output in JSON format is processed and converted into MASK format.

4. The method according to claim 1, characterized in that In step 2, DCGAN is a generative adversarial network that combines deep learning and convolutional neural networks. It consists of a generator and a discriminator. The structure of the generator is as follows: the input layer receives a random noise vector as input; the fully connected layer maps the input noise vector into a higher-dimensional feature vector; the transposed convolution layer gradually increases the spatial dimension of the feature map to restore the spatial resolution of the image; the batch normalization layer is used after each convolution layer to speed up the training and stabilize the learning process of the model; the last layer uses tanh as the activation function to limit the range of the output value to [-1, 1], and other layers use ReLU as the activation function; the output layer outputs a generated image with the same size and number of channels as the training data.

5. The method according to claim 4, characterized in that The DCGAN network discriminator is used to distinguish whether the input image is a real image from the training data or a fake image generated by the generator. The structure is as follows: the convolution layer extracts the features of the input image, and the convolution step size is set to 2; the batch normalization layer is used in all layers except the last layer; The LeakyReLU function is selected as the activation function; the fully connected layer is used to further process features and perform classification, and is used after the convolutional layer; the output layer is a fully connected layer with a sigmoid activation function, and the range of the output value is limited to [0,1], indicating the probability that the input image is a real image.

6. The method according to claim 1, characterized in that In step 3, MobileNetV4 is used to replace the original Xception network to enhance the encoder feature extraction capability. The original structure of the model framework is the semantic segmentation architecture of DeepLabV3+, which adopts an encoder-decoder structure. In the encoder part, it includes an improved Xception backbone network and a spatial pyramid pooling module. The depth-separable convolution operations of different channels in the Xception network model are used to segment multiple image features. The spatial pyramid pooling module is used at the end of the encoder to identify the multi-scale semantic features of the image. The decoder part performs 4x bilinear upsampling, feature fusion, 3*3 convolution and 4x bilinear upsampling operations in succession to complete the refinement of the boundaries of each target feature of the image and obtain the final feature segmentation result.

7. The method according to claim 1, characterized in that In step 5, the semantic segmentation model assigns each pixel in the two-dimensional time-frequency map to a specific category to distinguish different objects and boundaries in the image. The two-dimensional time-frequency map is input and a pixel-level classification map of the same size as the input image is output, in which each pixel corresponds to a category label, including background, normal speed signal and interference signal.

Images