Generative adversarial network-based vertical measurement ionogram trace segmentation method

Abstract

The invention discloses a vertical measurement ionogram trace segmentation method based on a generative adversarial network. The method comprises the following steps: S1, constructing a generator and a discriminator of the generative adversarial network; s2, preprocessing is carried out before the training data set is input into the network, maximum value normalization is carried out on each channel of an ionogram, the size of the ionogram is expanded to 512 * 512 through zero filling, and the obtained data set is divided into a training set and a test set; s3, inputting the ionograms of the training set into an encoder for feature extraction, inputting the obtained feature maps into a decoder for fusion of semantic information of different scales, and finally recovering the feature maps into segmentation result maps with the same size as the input ionograms; s4, the discriminator calculates the loss between different levels of feature maps by extracting the features of the input image so as to measure the difference between the real label and the segmentation result; and S5, the loss of the discriminator is fed back to the generator, and the generator adjusts network parameters according to the loss of the generator and the loss fed back by the discriminator.

Description

Technical Field

[0001] The present invention relates to the technical field of ionization map discrimination, and in particular to a vertical ionization map tracing segmentation method based on a generative adversarial network. Background Art

[0002] The ionospheric vertical detection method is the oldest means of ionospheric observation and research, and is still widely used today. Its basic principle is to use an ionospheric altimeter to transmit a swept high-frequency pulse wave in the vertical direction. When the operating frequency of the altimeter is less than the critical frequency of the ionosphere, the detection signal is reflected by the ionosphere and received by the ground receiver. By measuring the propagation delay of the corresponding frequency reflection echo, the ionospheric virtual height at each frequency point can be obtained. This data recorded by the ionospheric altimeter is the ionogram. Through the ionogram judgment, a variety of ionospheric parameter information can be obtained, and support can be provided for the inversion and reconstruction of the ionosphere, which plays an important role in the development of related sciences such as the ionosphere.

[0003] Ionogram interpretation can be done manually or automatically. Manual interpretation is done by experts based on their expertise. Although it has the highest accuracy, millions of ionograms are recorded every year around the world, and manual processing alone cannot meet the demand for real-time ionospheric information.

[0004] With the development of artificial intelligence technology, especially the success of deep learning in the field of image processing, neural networks have begun to be applied to ionogram tracing recognition and segmentation. In recent years, Xiao et al. proposed the DIAS model to handle complex ionogram discrimination tasks. Its performance is comparable to that of manual discrimination, but the segmented tracings have the problem of confusion between echo signals and background noise. Guo et al. improved the encoder Unet based on the DIAS model and introduced a multi-scale attention enhancement mechanism to improve the ionogram discrimination performance without significantly increasing the model parameters. However, most of the current deep learning tracing segmentation algorithms ignore the global information of the ionogram and are easily affected by the background noise of the complex ionogram, resulting in insufficient segmentation of the tracing. Summary of the invention

[0005] In view of the above-mentioned technical problems to be solved, the present invention provides a vertical measurement ionization map trace segmentation method based on a generative adversarial network, which has a stronger noise resistance ability for the judgment of ionization maps polluted by background noise, and the extracted ionization map trace is more refined.

[0006] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0007] A vertical ionization map tracing segmentation method based on a generative adversarial network comprises the following steps:

[0008] Step S1, construct the generator and discriminator of the generative adversarial network. The generator adopts the U-shaped network structure of encoder-decoder; the discriminator consists of six convolution modules CBL and a splicing layer. The convolution module CBL includes a 4×4 convolution layer, a batch normalization layer and a Leaky ReLU activation layer;

[0009] Step S2, preprocessing the original ionogram and the corresponding label map of the training data set before inputting them into the network, performing maximum normalization on each channel of the original ionogram, expanding the size of the original ionogram to a pixel size of 512×512 by zero padding, converting the background of the label map from black to white, and dividing the obtained data set into a training set and a test set;

[0010] Step S3, inputting the ionization map of the training set into the encoder of the generator for feature extraction, and inputting the obtained feature map into the decoder for fusing semantic information of different scales, and finally restoring the segmentation result map with the same size as the input ionization map;

[0011] Step S4, the discriminator receives the segmentation result image output by the generator and the label image of the training set, extracts the features of the two images through the convolution module CBL, doubles the number of channels, halves the size of the feature map, and calculates the loss between the two images to measure the difference;

[0012] In step S5, the loss of the discriminator is fed back to the generator, and the generator adjusts the network parameters according to the loss of the generator and the loss fed back by the discriminator to improve the generated segmented image.

[0013] As a further improvement of the above technical solution:

[0014] Preferably, in step S1, the encoder of the generator is composed of four encoding modules EM, each encoding module EM includes multiple residual modules, the residual module includes three convolution layers, and the convolution kernel sizes are 1×1, 3×3 and 1×1 respectively.

[0015] Preferably, in step S1, the decoder of the generator consists of four decoding modules and a concatenation layer, and the decoding module includes a 1×1 convolution layer, an upsampling fusion module, and two 3×3 convolution layers.

[0016] Preferably, in step S3, the ionogram of the training set is input into the encoder of the generator to extract multi-level features of the input ionogram through multiple residual modules, and the input ionogram features are gradually reduced to a deeper feature space through layer-by-layer convolution operations.

[0017] Preferably, in step S3, the feature map obtained by the encoder is input into the decoder to fuse semantic information of different scales. Each decoding module corresponds to the number of feature map channels transmitted by the encoding module through a 1×1 convolution layer. The upsampling fusion module receives two inputs, one is the feature map transmitted by the corresponding encoding module, and the other is the feature map obtained by upsampling of the previous encoding module. After addition, it is smoothed through two 3×3 convolution layers to generate a feature map containing semantic information of different scales; it is decoded by four decoding modules to restore the multi-scale semantic information to the final segmentation result.

[0018] Preferably, in step S1, the discriminator is composed of six convolution modules CBL and a concatenation layer, and the convolution module CBL includes a 4×4 convolution layer, a batch normalization layer and a Leaky ReLU activation layer.

[0019] Preferably, in step S3, a focus loss function is selected as the generator loss function, and the formula is as follows:

[0020]

[0021] Where y∈{±1} represents the true label category, p∈[0,1] represents the predicted probability of the class with y=1 output by the network, and FL(p t ) is the focus loss function, γ is the focus parameter, (1-p t ) γ is the modulation factor.

[0022] Preferably, in step S4, the loss between images is calculated by a multi-scale loss function, and the formula is as follows:

[0023]

[0024] Where x is the label image of the training set, x' is the segmentation result image generated by the generator, and f D () represents the hierarchical features extracted by the discriminator from the image. It represents the features extracted from the image at the i-th layer of the discriminator, and L represents the total number of layers of the discriminator network.

[0025] The vertical ionization map tracing segmentation method based on generative adversarial network provided by the present invention has the following advantages compared with the prior art:

[0026] The vertical ionogram tracing segmentation method based on generative adversarial network of the present invention uses a network composed of ResNet50 and FPN as a generator and a multi-layer CNN as a discriminator. For a given ionogram, the generator generates a segmentation result, and the discriminator provides feedback loss to the generator for adversarial training, thereby enhancing the segmentation performance of the generator. The discrimination method of the present invention can accurately and effectively perform tracing segmentation on the E, F1 and F2 layer echo tracings in the ionogram. Compared with the existing discrimination method, the discrimination anti-noise ability for the ionogram contaminated by background noise is stronger, and the extracted ionogram tracing is more refined, which can provide technical support for subsequent parameter extraction and ionospheric inversion, and is of great significance for real-time acquisition of ionospheric characteristics and structural information. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a model diagram of the present invention.

[0028] Figure 2 This is the generator encoder network structure diagram of the present invention.

[0029] Figure 3 It is a convolution residual module diagram of the present invention.

[0030] Figure 4 It is the identical residual module diagram of the present invention.

[0031] Figure 5 This is the generator decoder network structure diagram of the present invention.

[0032] Figure 6 This is the discriminator network structure diagram of the present invention.

[0033] Figure 7 (a) is the original ionization diagram

[0034] Figure 7 (b) is the label image judged manually

[0035] Figure 7 (c) is the image after image segmentation using the discrimination method of the present invention. DETAILED DESCRIPTION

[0036] The specific implementation of the present invention is described in detail below. It should be understood that the specific implementation described herein is The formula is only used to illustrate and explain the present invention and is not used to limit the present invention.

[0037] like Figures 1 to 6 As shown, the vertical ionization map tracing segmentation method based on generative adversarial network of the present invention uses a network combining ResNet50 and FPN as a generator and a multi-layer CNN as a discriminator. For a given ionization map, the generator generates a segmentation result, and the discriminator provides feedback loss to the generator for adversarial training, thereby enhancing the segmentation performance of the generator.

[0038] The vertical ionization map tracing segmentation method based on the generative adversarial network of the present invention comprises the following specific steps:

[0039] Step S1, construct the generator and discriminator of the generative adversarial network.

[0040] Generators such as Figure 2 As shown in the figure, a U-shaped network structure of encoder-decoder is adopted, with ResNet50 as the encoder and FPN as the decoder.

[0041] In this embodiment, the encoder ResNet50 of the generator includes four encoding modules, and there is an initial convolution layer before the four encoding modules, wherein the convolution kernel size of the initial convolution layer is 7×7, the step size is 2, the number of channels is 64, and a 3×3 maximum pooling layer is connected. The initial convolution layer reduces the size of the input ionization map by half, the number of channels becomes 64, and the number of network parameters is reduced. The 3×3 maximum pooling layer upsamples the feature map output by the initial convolution layer without changing the number of channels of the feature map, compresses the feature map while retaining the image features, and speeds up the operation of the neural network.

[0042] Each encoding module contains a different number of residual modules, which are divided into convolutional residual modules and identity residual modules. Skip connections are applied between residual modules to promote gradient flow and feature reuse.

[0043] The encoding modules EM1 and EM4 include three residual modules, the encoding module EM2 includes four residual modules, and the encoding module EM3 includes six residual modules. Each residual module includes three convolutional layers, and the convolution kernel sizes are 1×1, 3×3, and 1×1, respectively. The first residual module of the encoding modules EM2, EM3, and EM4 is a convolutional residual module, which adjusts the spatial dimension of the input by adding a convolutional layer with a stride of 2 or a 1×1 convolutional layer at the jump connection to achieve dimensional matching between the input and output. The subsequent residual modules are identity residual modules, and the input is directly mapped to the output through the jump connection to maintain the consistency of the spatial dimension and number of channels between the input and output. The convolutional residual module is as follows: Figure 3 As shown in Figure 4 As shown. The ReLU activation function formula is as follows:

[0044] ReLU(z)=max(z,0)

[0045] Among them, z is the input vector.

[0046] In this embodiment, the decoder FPN decoding module of the generator includes a 1×1 convolution layer, an upsampling fusion module and two 3×3 convolution layers. The resolution of the feature map is restored through the convolution and upsampling operations of the four decoding modules. The feature maps of different scales output by the four decoding modules are fused through the splicing layer to generate a feature map containing semantic information of different scales. The number of feature map channels is restored to the same as the input ionization map through 3×3 convolution to obtain the final segmentation result map.

[0047] like Figure 6 As shown in the figure, the discriminator consists of six convolution modules CBL and a concatenation layer. The convolution module CBL includes a 4×4 convolution layer, a batch normalization layer and a Leaky ReLU activation layer.

[0048] When training a generative adversarial network, the adversarial training between the generator and the discriminator makes the training unstable. The batch normalization layer can stabilize the training of the discriminator and increase the convergence speed. The Leaky ReLU activation layer avoids the zero gradient problem of the standard ReLU activation layer and can provide better gradient flow when processing complex image features. The formula of the Leaky ReLU activation function is as follows:

[0049]

[0050] Among them, z is the input vector and α is the negative slope coefficient.

[0051] Step S2, preprocess the training dataset before inputting it into the generator, perform maximum normalization on each channel of the ionogram, expand the image size to 512×512 by zero padding, and convert the image background from black to white. The obtained dataset is divided into a training set and a test set.

[0052] The data set used in this embodiment (which has been divided into training set and test set) is provided by the Institute of Geology and Geophysics, Chinese Academy of Sciences, which is 19910 ionograms recorded by the DPS4D ionospheric digital altimeter (114.4°E, 30.5°N) in a certain city from February 2018 to August 2018. Due to the changes in the configuration of the DPS4D ionospheric altimeter during the observation process, the sizes of the recorded ionograms are different, so the maximum value of each channel of the ionogram is normalized, the image size is expanded to 512×512 by zero padding, and the background of the label image of the training set is converted from black to white to facilitate the display of the segmentation effect. The data set disclosed by the institute is divided into training set and test set according to the recording time of the altimeter, where the ionograms recorded from February to July 2018 are the training set, and the ionograms recorded in August are the test set. The obtained data set is divided into training set and test set, the number of training sets is 16935, and the number of test sets is 2975.

[0053] Step S3, input the ionization map of the training set into the encoder of the generator for feature extraction, and input the obtained feature map into the decoder for fusing semantic information of different scales, and finally restore it to a segmentation result map of the same size as the input training set ionization map.

[0054] The ionization map of the training set is input into the encoder ResNet50 of the generator for feature extraction. The feature map obtained after the initial convolution layer is 256×256 pixels in size and 64 channels. The feature map C2 obtained by the encoding module EM1 is 128×128 pixels in size and 256 channels in number. The feature map C3 obtained by the encoding module EM2 is 64×64 pixels in size and 512 channels in number. The feature map C4 obtained by the encoding module EM3 is 32×32 pixels in size and 1024 channels in number. The feature map C5 obtained by the encoding module EM4 is 16×16 pixels in size and 2048 channels in number. C2, C3, C4 and C5 are input into the decoder FPN for the fusion of semantic information of different scales. First, the number of feature map channels is unified to 256 through the 1×1 convolution layer to obtain feature maps P2, P3, P4 and P5. The fusion between feature maps of different layers needs to ensure the same size dimension. In order to align the feature maps in the spatial dimension for pixel-by-pixel fusion, P3, P4, and P5 are upsampled to the same resolution as P2 through the nearest neighbor, and then P2, P3, P4, and P5 are spliced ​​together through the splicing layer to obtain a comprehensive feature map containing multi-scale semantic information with a channel number of 512. Finally, the spliced ​​feature map is restored to the same size as the input ionization map through bilinear upsampling, and the number of feature channels is reduced to 3 through 3×3 convolution, and the final segmentation result map is output through the sigmoid activation layer.

[0055] Since there is a class imbalance between the background noise pixels and the echo signal pixels in the ionization image, the focus loss function is selected as the generator loss function to alleviate it. The formula is as follows:

[0056]

[0057] Where y∈{±1} represents the true label category, p∈[0,1] represents the predicted probability of the class output by the network for y=1, and for ease of representation, p is used. t Instead of p, FL(p t ) is the focal loss function, where γ is the focusing parameter, (1-pt) γ The modulation factor can reduce the loss of easy-to-classify samples and make the model pay more attention to difficult-to-classify samples.

[0058] In step S4, the image input to the discriminator is the segmentation result image finally output by the generator, and the label image corresponding to the ionization image input to the generator in the training set. The features of the two images are extracted through the convolution module CBL, the number of channels is doubled, the size of the feature map is halved, and the loss between the two images is calculated to measure the difference.

[0059] The discriminator receives the segmentation result map output by the generator and the label map of the training set, and extracts the features of the two images through the convolution module, doubling the number of channels and halving the size of the feature map. The image input to the discriminator is 512×512 pixels in size and 3 channels in number. After the first convolution module, the feature map pixel size is 256×256 and the number of channels is 64; after the second convolution module, the feature map pixel size is 128×128 and the number of channels is 128; after the third convolution module, the feature map pixel size is 64×64 and the dimension is 256; after the fourth convolution module, the feature map pixel size is 32×32 and the dimension is 512; after the fifth convolution module, the feature map pixel size is 16×16 and the dimension is 512; after the sixth convolution module, the feature map pixel size is 16×16 and the dimension is 512. The feature map obtained by each convolution module is flattened into a vector of dimension 1, and they are spliced ​​together along the feature dimension through the splicing layer. Calculate the multi-scale L1 loss between the training set label and the output of the generator’s segmentation result input to the discriminator to measure the difference between the two images;

[0060] The calculation formula of the multi-scale L1 loss function of the discriminator is as follows:

[0061]

[0062] Where x is the label image of the training set, x' is the segmentation result image generated by the generator, and f D () represents the hierarchical features extracted by the discriminator from the image. It represents the features extracted from the image at the i-th layer of the discriminator, and L represents the total number of layers of the discriminator network.

[0063] In step S5, the loss of the discriminator is fed back to the generator, and the generator adjusts the network parameters according to the loss of the generator and the loss fed back by the discriminator to improve the generated segmented image.

[0064] Set the number of iterations of the generative adversarial network to 160000, the optimizer to RMSprop, and the learning rate to 0.0001. The generator and the discriminator are alternately optimized through an adversarial training strategy. The generator uses the multi-scale L1 loss fed back by the discriminator and the binary focal loss of the generator to adjust the network parameters to generate better segmentation results. The adversarial training strategy: The generator and the discriminator are alternately trained to optimize different objective functions. The generator needs to minimize the generator's loss function, and the discriminator needs to maximize the discriminator's loss function. The way to adjust the network parameters is: when the discriminator's loss is large, it means that the segmentation result generated by the generator in this training is very different from the label, and the network parameters need to be adjusted to reduce the discriminator loss.

[0065] The training process of the generative adversarial network is as follows: first input the ionization map and its corresponding label, fix the generator, input the original ionization map to generate the segmentation result. The discriminator receives the label and the segmentation result, calculates the multi-scale L1 loss between the two, and back-propagates to update the discriminator parameters. The discriminator is fixed, the generator receives the ionization map to generate the segmentation result, calculates the binary focal loss of the label and the segmentation result, and back-propagates with the multi-scale loss fed back by the discriminator to update the generator parameters. When the model iteration number reaches 160,000, stop training, save the model, input the ionization map of the test set into the generator, generate the segmentation result map, and compare it with the label to verify the model segmentation performance.

[0066] The discriminant method of the present invention utilizes the powerful feature extraction capability and large data set processing capability of deep learning, adopts adversarial training strategy, and alternately optimizes the generator and the discriminator, so that the difference between the segmentation result generated by the generator and the real label becomes smaller and smaller, and a fine ionization map is obtained. The effect can be seen from Figure 7 The comparison shows that Figure 7 Schematic diagram of images before and after ionization diagram determination using the method of the present invention in an embodiment of the present invention; wherein Figure 7 (a) is the original ionization diagram; Figure 7 (b) is the label image judged manually; Figure 7 (c) is an image after image segmentation using the method of the present invention, Figure 7 (c) It can be seen that the method described in the present invention can effectively and accurately separate the signals of each layer on the ionogram, which is convenient for the next step of ionospheric parameter extraction.

[0067] The above implementation cases are only preferred embodiments of the present invention and are not intended to limit the present invention in any form. Although the present invention has been disclosed as above with preferred embodiments, they are not intended to limit the present invention. Therefore, any simple modification, equivalent changes and modifications made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A vertical ionization map tracing segmentation method based on generative adversarial network, characterized in that: The following steps are involved: Step S1, constructing the generator and discriminator of the generative adversarial network. The generator adopts the U-shaped network structure of encoder-decoder; the discriminator consists of six convolution modules CBL and a concatenation layer. The convolution module CBL includes a 4×4 convolution layer, a batch normalization layer and a Leaky ReLU activation layer; Step S2, before inputting the original ionogram and the corresponding label map of the training data set into the network, data preprocessing is performed, each channel of the original ionogram is normalized to the maximum value, the size of the original ionogram is expanded to a pixel size of 512×512 by zero padding, the background of the label map is converted from black to white, and the obtained data set is divided into a training set and a test set; Step S3, inputting the ionization map of the training set into the encoder of the generator for feature extraction, and inputting the obtained feature map into the decoder for fusing semantic information of different scales, and finally restoring the segmentation result map with the same size as the input ionization map; Step S4, the discriminator receives the segmentation result image output by the generator and the label image of the training set, extracts the features of the two images through the convolution module CBL, doubles the number of channels, halves the size of the feature map, and calculates the loss measurement difference between the two images; In step S5, the loss of the discriminator is fed back to the generator, and the generator adjusts the network parameters according to the loss of the generator and the loss fed back by the discriminator to improve the generated segmented image.

2. The method for segmenting vertical ionization map traces based on a generative adversarial network according to claim 1, characterized in that: In step S1, the encoder of the generator is composed of four encoding modules EM, each encoding module EM includes multiple residual modules, the residual module includes three convolution layers, and the convolution kernel sizes are 1×1, 3×3 and 1×1 respectively.

3. The method for segmenting vertical ionization map traces based on a generative adversarial network according to claim 1, characterized in that: In step S1, the decoder of the generator consists of four decoding modules and a concatenation layer, and the decoding module includes a 1×1 convolution layer, an upsampling fusion module, and two 3×3 convolution layers.

4. The method for segmenting vertical ionization map traces based on a generative adversarial network according to claim 2, characterized in that: In step S3, the ionogram of the training set is input into the encoder of the generator, and the multi-level features of the input ionogram are extracted through multiple residual modules. Through layer-by-layer convolution operations, the input ionogram features are gradually reduced to a deeper feature space.

5. The method for segmenting vertical ionization map traces based on generative adversarial networks according to claim 3, characterized in that: In step S3, the feature map obtained by the encoder is input into the decoder to fuse semantic information of different scales. Each decoding module corresponds to the number of feature map channels transmitted by the encoding module through a 1×1 convolution layer. The upsampling fusion module receives two inputs, one is the feature map transmitted by the corresponding encoding module, and the other is the feature map obtained by upsampling of the previous encoding module. After addition, it is smoothed through two 3×3 convolution layers to generate a feature map containing semantic information of different scales; it is decoded by four decoding modules to restore the multi-scale semantic information to the final segmentation result.

6. The method for segmenting vertical ionization map traces based on a generative adversarial network according to claim 3, characterized in that: In the step S1, the discriminator is composed of six convolution modules CBL and a concatenation layer, and the convolution module CBL includes a 4×4 convolution layer, a batch normalization layer and a Leaky ReLU activation layer.

7. The method for segmenting vertical ionization map traces based on a generative adversarial network according to claim 1, characterized in that: In step S3, the focus loss function is selected as the generator loss function, and the formula is as follows: Where y∈{±1} represents the true label category, p∈[0,1] represents the predicted probability of the class with y=1 output by the network, and FL(p t ) is the focus loss function, γ is the focus parameter, (1-p t ) γ is the modulation factor.

8. The method for segmenting vertical ionization map traces based on a generative adversarial network according to claim 1, characterized in that: In step S4, the loss between images is calculated by a multi-scale loss function, and the formula is as follows: Where x is the label image of the training set, x' is the segmentation result image generated by the generator, and f D () represents the hierarchical features extracted by the discriminator from the image. It represents the features extracted from the image at the i-th layer of the discriminator, and L represents the total number of layers of the discriminator network.

Images