A method and device for inversion of all-round non-uniform low-altitude waveguide profile based on multi-head and multi-tail VIT model

Abstract

The present invention discloses a method for inverting the profile of an all-round non-uniform low-altitude waveguide based on a multi-head multi-tail VIT model, comprising the following steps: step 1, for the evaporation waveguide height information in the non-uniform evaporation waveguide region, the optimal result of the non-uniform evaporation waveguide spatial simulation is obtained; step 2, the calculated sea clutter power map and the non-uniform evaporation waveguide height map are used to form a training data set required for a deep learning inversion model; step 3, a network model based on a deep neural network multi-head multi-tail VIT model is established; step 4, a non-uniform evaporation waveguide profile spatial variation map is output. The method and device disclosed by the present invention optimize the original Transformer network, and propose a multi-head multi-tail network structure to construct a non-linear mapping of the full spatial variation map of the non-uniform evaporation waveguide and the sea clutter power map.

Description

Technical Field

[0001] The invention belongs to the field of ocean atmosphere remote sensing, and particularly relates to an omnidirectional non-uniform low-altitude waveguide profile inversion method and device based on a multi-head and multi-tail VIT model in the field. Background Art

[0002] The method of inverting low-altitude waveguides based on sea clutter is currently a relatively reliable atmospheric waveguide remote sensing method that can meet the requirements of waveguide structures. However, it currently has the following main shortcomings: First, the sea clutter-to-noise ratio is low under low sea conditions, the effective sea clutter distance range is small, and insufficient data sources lead to low atmospheric waveguide inversion accuracy and limited inversion areas; second, the current technology only inverts waveguide parameters at a single azimuth and different distances. The method of obtaining regional waveguides by splicing inversion results at different azimuths is difficult to meet real-time requirements, and the gradual change law of waveguide parameters at different azimuths is not sufficiently considered.

[0003] Foreign scholars have used a hybrid algorithm combining RBF and GA to achieve the inversion of evaporation duct profiles. However, the above inversion methods will bring a lot of time and space complexity in each inversion iteration process, and require a lot of computing resources. In order to improve the efficiency of the model, a deep neural network is proposed to achieve the inversion of non-uniform evaporation duct profiles. However, when the current traditional deep neural network is used for the inversion of three-dimensional non-uniform evaporation duct profiles with spatial characteristics, it will bring more model parameters to the model, affecting the efficiency of the actual model inversion. Summary of the invention

[0004] The technical problem to be solved by the present invention is that the traditional inversion models RBF and GA are inefficient, and when the traditional deep neural network is used for the inversion of a three-dimensional non-uniform evaporation duct profile with spatial characteristics, it will bring higher model parameters to the model, resulting in a large gap between the inversion result and the actual inversion result. An inversion method and device for an all-round non-uniform evaporation duct based on a multi-head and multi-tail VIT deep learning model are provided, which realizes the efficient inversion of regional sea clutter data areas and supports the evaporation duct inversion research based on radar sea clutter data.

[0005] The present invention adopts the following technical solution:

[0006] An all-round non-uniform low-altitude waveguide profile inversion method based on a multi-head and multi-tail VIT model, the improvement of which is that it includes the following steps:

[0007] Step 1: for the evaporation waveguide height information in the non-uniform evaporation waveguide area, the change of the evaporation waveguide height is used as the constraint distance, and according to the reference distance, a two-dimensional Markov chain calculation model is used to conduct experiments on different values ​​to obtain the optimal result of the non-uniform evaporation waveguide space simulation;

[0008] Step 2: Substitute the full-space non-uniform evaporation duct height profile generated by the coupled two-dimensional Markov chain model into the parabolic equation and the radar sea clutter power calculation formula to obtain the corresponding 360° sea clutter power map data. The calculated sea clutter power map and the non-uniform evaporation duct height map constitute the training data set required for the deep learning inversion model.

[0009] Step 3: Establish a comprehensive non-uniform evaporation waveguide inversion network model based on a deep neural network multi-head multi-tail VIT model:

[0010] Build the Transformer encoder and Transformer decoder for feature extraction and the tails module for restoring the feature map to output;

[0011] Contrastive learning is introduced to learn common features. The loss function L of contrastive learning is constrastive as follows:

[0012]

[0013]

[0014] In the above formula, i = {1,...,N}, N is the number of Transformer decoders, For input x j After the Transformer decoder outputs two feature blocks, x j is selected from a training image set X, X = {x1,...,x B}, B is the total number of inputs,

[0015] The overall loss function L IPT for:

[0016] L IPT =λ·L constrastive +L supervised

[0017] In the above formula, the parameter λ is used to adjust the contrast loss function L constrastive And the supervision loss function L supervised The weight of

[0018] Step 4: When training the model, select sigmoid as the activation function, RMSE as the loss function, and RMSprop as the optimization function; after using these training data and parameters to train the deep neural network model, input the 360° sea clutter power, and the corresponding output is the spatial variation map of the non-uniform evaporation waveguide profile predicted by the model.

[0019] Furthermore, in step 1, for the evaporation duct height information at 8 distances in the 100×100 km non-uniform evaporation duct area, the changes in the evaporation duct height at distances of 20, 30, 50, 60, 70, and 100 km were used as constraint distances, and 40 and 80 were used as reference distances. A two-dimensional Markov chain calculation model was used to conduct 400 experiments on different values ​​to obtain the optimal result of the non-uniform evaporation duct spatial simulation.

[0020] Furthermore, in step 2, the radar system parameters used for calculation are: radar frequency is X-band, antenna height is 10 meters, beam width is 0.7°, and horizontal polarization is adopted.

[0021] Furthermore, in step 2, the input is 1400 sets of sea clutter power data at 0km-100km, with a distance interval of 1km; the output is 1400 evaporation waveguide area change matrices at 0km-100km.

[0022] Furthermore, in step 4, by training the model, the number of iterations is 500, and the deep learning network includes a Transformer encoder, a Transformer decoder, and a multi-tail data reconstruction deep learning network.

[0023] An all-round non-uniform low-altitude waveguide profile inversion device based on a multi-head and multi-tail VIT model, the improvement of which lies in: being implemented using the above method steps.

[0024] The beneficial effects of the present invention are:

[0025] The method and device disclosed in the present invention are based on a multi-head and multi-tail attention mechanism VIT (MM-VIT) model, which takes into account the nonlinear mapping relationship between the non-uniform evaporation waveguide map and the sea clutter echo power map, optimizes the original Transformer network, and proposes a multi-head and multi-tail Transformer network structure to construct a nonlinear mapping between the full-space variation map of the non-uniform evaporation waveguide and the sea clutter power map.

[0026] In order to verify the effectiveness of the method of the present invention for inversion of non-uniform evaporation duct profiles, two sets of simulated non-uniform evaporation duct heights were used for verification. The accuracy of the method of the present invention exceeded the most advanced inversion method available.

[0027] The trained omnidirectional non-uniform atmospheric waveduct inversion model is transferred to the ship navigation radar. From the echo imaging, it can be observed whether there is interference and clutter in the image direction, and whether there is atmospheric waveduct echo drift in the image zero position. Through the display of the echo signal, it can be preliminarily judged whether there is a problem with the atmospheric waveduct. It provides an effective way to invert the temporal and spatially varying non-uniform evaporation waveduct, which is beneficial to the study of the evaporation waveduct profile based on sea clutter power data. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic flow diagram of the method of the present invention;

[0029] Figure 2 This is the network architecture diagram of the multi-head and multi-tail VIT model;

[0030] Figure 3 are the specific parameters of the long-head and long-tail VIT model;

[0031] Figure 4 This is a comparison chart of the inversion accuracy between the method of the present invention and methods using DNN, CNN, VGGNet, ResNet, MobileNet, Transformer, and FCCT-Transformer models. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0033] In the process of inverting the actual full-space non-uniform evaporation waveguide, due to the limited amount of non-uniform evaporation waveguide data in the low-altitude range of the sea, in order to improve the accuracy and effectiveness of the spatial inversion of the full-space non-uniform evaporation waveguide, this application migrates the Transformer model with a self-attention mechanism to the inversion task of the non-uniform waveguide evaporation waveguide profile. Compared with the convolution operation of the traditional CNN, the self-attention mechanism is not limited by local interactions, and can not only mine the dependencies within the distance range of the non-uniform atmospheric waveguide, but also perform parallel calculations. It can learn the most appropriate inductive bias according to different task objectives, and has achieved good results in the inversion task of the non-uniform atmospheric waveguide.

[0034] Example 1. This example discloses a method for inverting the omnidirectional non-uniform low-altitude waveguide profile based on a multi-head multi-tail VIT model. The omnidirectional power data of sea clutter is input into a multi-head multi-tail VIT deep learning network to extract the omnidirectional non-uniform evaporation waveguide profile parameters. The nonlinear mapping relationship between the omnidirectional radar echo data of sea clutter and the regional non-uniform evaporation waveguide profile is obtained by training, so as to realize the inversion of radar sea clutter to non-uniform evaporation waveguide. Figure 1 As shown, the following steps are included:

[0035] Step 1: The non-uniform variation of the offshore evaporation duct profile is caused by the random variation of atmospheric factors in space. This variation is simulated using a coupled two-dimensional Markov process.

[0036] According to the evaporation waveguide height information in the non-uniform evaporation waveguide area, the change of the evaporation waveguide height is used as the constraint distance. According to the reference distance, a two-dimensional Markov chain calculation model is used to conduct experiments on different values ​​to obtain the optimal result of the non-uniform evaporation waveguide spatial simulation.

[0037] For the evaporation duct height information at 8 distances in a 100×100 km non-uniform evaporation duct area, the changes in the evaporation duct height at distances of 20, 30, 50, 60, 70, and 100 km were used as constraint distances, and 40 and 80 were used as reference distances. A two-dimensional Markov chain calculation model was used, and 400 experiments were conducted on different values ​​to obtain the optimal result of the non-uniform evaporation duct spatial simulation.

[0038] Step 2: Substitute the full-space non-uniform evaporation waveguide height profile generated by the coupled two-dimensional Markov chain model into the three-dimensional parabolic equation and the radar sea clutter power calculation formula to obtain the corresponding 360° all-round sea clutter power map data. The calculated sea clutter power map and the non-uniform evaporation waveguide height map are used to form the training data set required for establishing the deep neural network multi-head and multi-tail VIT inversion model.

[0039] The radar system parameters used in the calculation are: radar frequency is X-band, antenna height is 10 meters, beam width is 0.7°, and horizontal polarization is used.

[0040] The input is 1400 sets of sea clutter power data at 0km-100km (i.e. the input layer is 1400 sets of radar sea clutter power echo maps in a non-uniform atmospheric duct environment), with a distance interval of 1km; the output is 1400 0km-100km evaporation duct area change matrices (i.e. the output layer is 1400 sets of non-uniform evaporation duct height change maps).

[0041] Step 3: Establish a comprehensive non-uniform evaporation waveguide inversion network model based on a deep neural network multi-head multi-tail VIT model:

[0042] Building a deep learning inversion model to realize non-uniform evaporation waveguide profiles requires three tasks: building a Transformer encoder and a Transformer decoder for feature extraction, and a tails module that restores the feature map to output;

[0043] Contrastive learning is introduced to learn universal features so that pre-trained models can be used for unknown learning tasks. The goal of contrastive learning is to minimize the distance between features from the same image while maximizing the distance from different images. The loss function L of contrastive learning is constrastive as follows:

[0044]

[0045]

[0046] In the above formula, i = {1,...,N}, N is the number of Transformer decoders, For input x j After the Transformer decoder outputs two feature blocks, x j is selected from a training image set X, X = {x1,...,x B}, B is the total number of inputs, Represents the cosine similarity of two vectors, and the goal is to minimize the distance between feature patches from the same image while maximizing the distance between feature patches from different images.

[0047] In order to make full use of the supervision and self-supervision information, the overall loss function L of this model is IPT for:

[0048] L IPT =λ·L constrastive +L supervised

[0049] In the above formula, the parameter λ is used to adjust the contrast loss function L constrastive And the supervision loss function L supervised The weight of

[0050] Step 4: The network model constructed is based on the Python pytorch framework. When training the model, sigmoid is selected as the activation function, RMSE is selected as the loss function, and RMSprop is selected as the optimization function. After using these training data and parameters to train the deep neural network model, the 360° sea clutter power is input, and the corresponding output is the spatial variation map of the non-uniform evaporation waveguide profile predicted by the model.

[0051] During the training process in step 4, the training parameters and accuracy are continuously adjusted and optimized. If the requirements are not met, the network hyperparameters are adjusted. Through the training of the model, the number of iterations is 500 times. The deep learning network includes the Transformer encoder, the Transformer decoder and the multi-tail data reconstruction deep learning network. 500 groups of data are generated to form a test set for verifying the accuracy of the network model.

[0052] In order to select the best parameters to train the deep neural network of sea clutter inversion evaporation duct, it is necessary to analyze the factors affecting the accuracy of the inversion results. First, it is necessary to establish a test set to quantitatively evaluate the quality of the established network model. For the inversion of evaporation duct height, this embodiment extracts 200 groups of different evaporation duct parameters in the range of 0-100km, and generates 200 groups of sea clutter power data to form a test set. These 200 groups of sea clutter power are respectively input into the established model, and the inversion results are compared with the actual evaporation duct height corresponding to the test set, and the accuracy is calculated to judge the quality of the established model. By increasing the number of hidden layers and the number of iterations, the accuracy will continue to improve. This is because each iteration process is a process in which the deep neural network automatically adjusts the network weight parameters according to the loss function value. The value of the loss function will continue to decrease with the change of network parameters, and the increase in the number of hidden layers represents the increase in the number of neurons. The deep neural network can learn more training data features, thereby more accurately fitting the nonlinear relationship between sea clutter power and evaporation duct parameters, and the inverted evaporation duct height result is closer to the target true value. This embodiment uses 1400 sets of training data and iterates 1000 times to train the 360° sea clutter inversion full-space non-uniform evaporation waveguide profile parameter model. The network architecture of the model is as follows: Figure 2 Specific parameters are shown in Figure 3 shown.

[0053] In order to verify the effectiveness of the model proposed in this method for the inversion of non-uniform evaporation waveguide profiles (the accuracy of the model), two sets of full-space sea clutter power data sets were used for actual measurement verification. The test results show that compared with the fully connected network, the convolutional neural network achieved better inversion results. In the convolutional neural network, the method of the present invention uses a 3×3 convolution kernel for the convolution operation of the network model. VGGNet16, as a variant model of the convolutional neural network, has been effectively improved in terms of the accuracy and effectiveness of the inversion model. Due to the introduction of the residual network module, the parameters of the ResNet network and the complexity parameters of the model are further improved. Although the emergence of the residual network module can optimize the accuracy of the model inversion and the parameters of the model. However, too many stacks and network layers will still bring a lot of computing time. For the MobileNet network, due to its network structure based on depth-level separable convolution, different convolution kernels are used for each input channel, which can significantly reduce the model parameters and complexity. Although the MobileNet network can significantly reduce the parameters of the model, for a large amount of inversion data of non-uniform evaporation waveguides, the adaptation performance of the MobileNet network for a large amount of data is not as ideal as that of the Transformer model. Due to the superposition of its multi-head attention mechanism, the Transformer model can expand the receptive field of the image and obtain more network environment information. Since the self-attention mechanism has more feature aggregation capabilities for the model, the globality of the model is stronger. FCCT-Transformer is improved on the basis of the Transformer network, replacing the traditional convolution kernel with the dilated causal convolution, and adding a focus mechanism to optimize the Transformer network, which greatly reduces the accuracy of the inversion model running the model. Figure 4 As shown, the model proposed by the method of the present invention is a variant model of the Transformer network, which adopts a multi-head and multi-tail model design and achieves the best inversion result.

[0054] This embodiment also discloses an all-round non-uniform low-altitude waveguide profile inversion device based on a multi-head and multi-tail VIT model, which is implemented using the above-mentioned method steps.

Claims

1. A method for inverting the profile of an all-round non-uniform low-altitude waveguide based on a multi-head and multi-tail VIT model, characterized in that: The steps include: Step 1: for the evaporation waveguide height information in the non-uniform evaporation waveguide area, the change of the evaporation waveguide height is used as the constraint distance, and according to the reference distance, a two-dimensional Markov chain calculation model is used to conduct experiments on different values ​​to obtain the optimal result of the non-uniform evaporation waveguide space simulation; Step 2: Substitute the full-space non-uniform evaporation duct height profile generated by the coupled two-dimensional Markov chain model into the parabolic equation and the radar sea clutter power calculation formula to obtain the corresponding 360° sea clutter power map data. The calculated sea clutter power map and the non-uniform evaporation duct height map constitute the training data set required for the deep learning inversion model. Step 3: Establish a comprehensive non-uniform evaporation waveguide inversion network model based on a deep neural network multi-head multi-tail VIT model: Build the Transformer encoder and Transformer decoder for feature extraction and the tails module for restoring the feature map to output; Contrastive learning is introduced to learn common features. The loss function L of contrastive learning is constrastive as follows: In the above formula, i = {1,...,N}, N is the number of Transformer decoders, For input x j After the Transformer decoder outputs two feature blocks, x j is selected from a training image set X, X = {x1,...,x B }, B is the total number of inputs, The overall loss function L IPT for: L IPT =λ·L constrastive +L supervised In the above formula, the parameter λ is used to adjust the contrast loss function L constrastive And the supervision loss function L supervised The weight of Step 4: When training the model, select sigmoid as the activation function, RMSE as the loss function, and RMSprop as the optimization function; after using these training data and parameters to train the deep neural network model, input the 360° sea clutter power, and the corresponding output is the spatial variation map of the non-uniform evaporation waveguide profile predicted by the model.

2. The omnidirectional non-uniform low-altitude waveguide profile inversion method based on the multi-head and multi-tail VIT model according to claim 1 is characterized by: In step 1, for the evaporation duct height information at 8 distances in the 100×100 km non-uniform evaporation duct area, the changes in the evaporation duct height at distances of 20, 30, 50, 60, 70, and 100 km were used as constraint distances, and 40 and 80 were used as reference distances. A two-dimensional Markov chain calculation model was used to conduct 400 experiments on different values ​​to obtain the optimal result of the non-uniform evaporation duct spatial simulation.

3. The omnidirectional non-uniform low-altitude waveguide profile inversion method based on the multi-head and multi-tail VIT model according to claim 1 is characterized in that: In step 2, the radar system parameters used for calculation are: radar frequency is X-band, antenna height is 10 meters, beam width is 0.7°, and horizontal polarization is used.

4. The omnidirectional non-uniform low-altitude waveguide profile inversion method based on the multi-head and multi-tail VIT model according to claim 1 is characterized in that: In step 2, the input is 1400 sets of sea clutter power data at 0km-100km, with a distance interval of 1km; the output is 1400 evaporation waveguide area change matrices at 0km-100km.

5. The omnidirectional non-uniform low-altitude waveguide profile inversion method based on the multi-head and multi-tail VIT model according to claim 1 is characterized in that: In step 4, by training the model, the number of iterations is 500, and the deep learning network includes a Transformer encoder, a Transformer decoder, and a multi-tail data reconstruction deep learning network.

6. An all-round non-uniform low-altitude waveguide profile inversion device based on a multi-head and multi-tail VIT model, characterized by: The method steps described in claim 1 are used to achieve this.

Images