A VHF signal propagation delay fitting method based on layer-by-layer greedy pre-training structure
By introducing layer-by-layer greedy pre-training structure and multi-head attention mechanism in deep neural networks, the gradient disappearance problem in VHF signal propagation delay fitting is solved, the training efficiency and generalization ability of the model are improved, and the sea propagation is achieved The precise fitting of signal propagation delay in the media improves the accuracy of land-based navigation positioning.
Patent Information
- Application Number
- CN202411215234.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-02
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-09-02
AI Technical Summary
The prior art has gradient vanishing problems when fitting the VHF signal propagation delay, resulting in low training efficiency and poor generalization capabilities of deep neural networks, making it difficult to achieve high-precision land-based navigation positioning.
Deep neural network based on layer-by-layer greedy pre-trained structure is adopted, and the feature learning of each hidden layer is learned by layer-by-layer greedy training structure to alleviate the problem of gradient vanishing, and combine the multi-head attention mechanism and autoencoder structure to improve the model's supervision. Efficiency and generalization capabilities of the fine-tuning stage.
It effectively alleviates the problem of gradient vanishing, improves the efficiency and generalization ability of the fitting model, realizes accurate fitting of VHF signal propagation delay, and improves the positioning accuracy of the land-based navigation system.
Smart Images

Figure CN118734078B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of radio navigation, relates to the cross-application subject of land-based positioning signal processing and radio wave propagation, and specifically relates to a VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure. Background Art
[0002] In order to obtain real-time ship information and implement navigation supervision, maritime traffic management personnel usually rely on the vessel traffic management system, namely VTS. However, VTS has many functions and responsibilities, and the workload is heavy, which is prone to problems such as inadequate order monitoring and information services, and delayed command. For this reason, a system that automatically obtains ship information and navigation status came into being, namely the Automatic Identification System (AIS). The positioning, navigation and timing information of AIS is provided by its embedded GNSS receiver, and GNSS signals have natural vulnerabilities. When GNSS is interfered with and its function fails, the shipboard equipment cannot obtain correct positioning, navigation and timing information, threatening navigation safety. For this reason, research should be conducted to enable AIS to have land-based navigation functions independent of satellite navigation.
[0003] Positioning accuracy is one of the important indicators to measure the availability of land-based navigation systems. There are three main sources of error that determine the positioning accuracy of land-based navigation systems: the transmitting system, the propagation delay of the signal in the propagation medium, and the receiving system. The propagation delay of the signal in the channel is affected by the propagation characteristics of the signal and changes with the weather. The weather at sea is complex and changeable, so the fitting of the propagation channel delay is complex and challenging.
[0004] Existing technologies usually use the nonlinear modeling and feature learning capabilities of deep neural networks to accurately fit signal propagation delays, but as the number of deep neural network layers increases, the gradient vanishing problem becomes more obvious. Summary of the invention
[0005] In view of the shortcomings of the prior art, the present invention provides a VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure. The present invention introduces a layer-by-layer greedy pre-training structure in a deep neural network, and propagates the gradient between each layer by learning the features of each hidden layer, effectively alleviating the gradient vanishing problem, thereby improving the efficiency of the fitting model in the subsequent supervised fine-tuning stage.
[0006] The technical means adopted by the present invention are as follows:
[0007] A VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure comprises the following steps:
[0008] Acquiring historical VHF signal data, wherein the historical VHF signal data includes sea surface meteorological factor impact data and corresponding VHF signal propagation delay data;
[0009] A VHF signal propagation delay fitting model is trained based on the historical VHF signal data. The VHF signal propagation delay fitting model adopts a four-layer deep neural network structure with multiple inputs and a single output and a dual hidden layer structure. The VHF signal propagation delay fitting model first adopts a layer-by-layer greedy training structure for unsupervised pre-training, and then fine-tunes the learning rate, network weights and bias of each layer of the model through supervised overall training; the layer-by-layer greedy training structure includes four autoencoder structures, and the four autoencoder structures correspond to four levels of the VHF signal propagation delay fitting model, respectively. Each autoencoder structure includes an encoder and a decoder, and the encoder is used to adjust the dimension of the input sample data to the input dimension of the corresponding level of the VHF signal propagation delay fitting model, and the decoder is used to reconstruct the feature vector output by the encoder back to the dimension of the input sample data;
[0010] The influence data of the current sea surface meteorological factors are obtained, the trained VHF signal propagation delay fitting model is output, and the model output is obtained as the VHF signal propagation delay data fitting result.
[0011] Furthermore, the sea surface meteorological factors influencing data include sea water temperature, humidity, air pressure, air temperature, wind speed, wind direction and tidal water level.
[0012] Furthermore, the VHF signal propagation delay fitting model includes an input layer, a first hidden layer, a second hidden layer and an output layer, wherein:
[0013] The input layer uses a gated recurrent unit to extract intrinsic correlation features between input data;
[0014] The first hidden layer introduces a multi-head attention mechanism to further extract data features;
[0015] The second hidden layer is a fully connected layer for realizing multi-feature integration of data;
[0016] The output layer adopts a fully connected layer to output the VHF signal propagation delay fitting result.
[0017] Furthermore, the input layer extracts the intrinsic correlation features between the input data, including:
[0018] First initialize the hidden state , and then for each group of vectors in the input sample matrix, the time step Do the following:
[0019] a. Calculate reset gate : Calculate reset gate use Activation function, The calculation formula is:
[0020]
[0021] In the formula, is the weight matrix, For the current Enter data at all times. For the previous Always hide the status, is the bias vector, is the Sigmoid activation function;
[0022] b. Calculate the update gate :Calculate the update gate use activation, The calculation formula is:
[0023]
[0024] In the formula, is the weight matrix, For the current Enter data at all times. For the previous Always hide the status, is the bias vector, is the Sigmoid activation function;
[0025] c. Calculate candidate hidden states : Candidate hidden state The calculation adopts Activation function, The calculation formula is:
[0026]
[0027] In the formula, is the weight matrix, for Reset the gate signal at all times. For the previous Always hide the status, is the bias vector;
[0028] d. Calculate the current hidden state : The calculation formula is:
[0029]
[0030] In the formula, for Update the door signal at all times. For the previous Always hide the status, is the candidate hidden state at the current moment.
[0031] Furthermore, the data features are further extracted through the first hidden layer, including:
[0032] The hidden state obtained from the input layer is transformed into a query vector, a key vector and a value vector through a linear transformation; a set of attention weight values is calculated for each head in the multi-head attention mechanism, and the weight values are used to perform weighted summation on the query vector, key vector and value vector respectively; the weighted summed query vector, key vector and value vector are combined into a high-dimensional feature vector containing all the weighted summed query vectors, key vectors and value vectors.
[0033] Furthermore, the input sample data of the first hidden layer and the output layer are normalized to the same scale, and the normalization steps are as follows:
[0034] Assume The net input of the neurons in the layer is , The mean and variance of are:
[0035]
[0036]
[0037] In the formula, is the activation vector length, which is the number of neurons in this layer. For the Layer neurons, Net input The mean of Net input The variance of , from which we can get the net input of layer normalization for:
[0038]
[0039] In the formula, is a numerical stability parameter, and its value is ;
[0040] Add scaling and translation parameters to the normalized net input data. for:
[0041]
[0042] in, is the scaling parameter, is the translation parameter.
[0043] Furthermore, the VHF signal propagation delay fitting model adopts a layer-by-layer greedy training structure for unsupervised pre-training, including:
[0044] The training sample data is input into the first autoencoder in the form of feature vectors, and the encoder in the first autoencoder structure is trained in an unsupervised manner to obtain the output feature vector. , For the encoder output feature vector dimension in the first autoencoder structure, As the decoder input data in the first autoencoder, it is used to reconstruct the data dimension so that the output data dimension of the decoder is consistent with the original input data dimension, by minimizing the decoder input data and decoder output data The loss function is used to achieve the optimal reconstruction purpose, while preserving the weights and biases of each pre-training;
[0045] Will As the input data of the second autoencoder, the encoder in the second autoencoder is trained in an unsupervised manner to obtain the output features , For the encoder output feature vector dimension in the second autoencoder structure, As the decoder input data in the second autoencoder, it is used to reconstruct the data dimension so that the output data dimension of the decoder is consistent with the input data dimension of the second encoder, and the output sample is reconstructed using the minimum mean square error function;
[0046] Will As the input data of the third autoencoder, the encoder in the third autoencoder is trained in an unsupervised manner to obtain the output features , o is the dimension of the encoder output feature vector in the third self-encoder structure, As the decoder input data in the third autoencoder, it is used to reconstruct the data dimension so that the output data dimension of the decoder is consistent with the input data dimension of the third encoder, and the output sample is reconstructed using the minimum mean square error function;
[0047] Will As the input data of the fourth autoencoder, the encoder in the fourth autoencoder is trained in an unsupervised manner to obtain the output feature , p For the encoder output feature vector dimension in the fourth autoencoder structure, As the decoder input data in the fourth autoencoder, it is used to reconstruct the data dimension so that the output data dimension of the decoder is consistent with the input data dimension of the fourth encoder, and the output sample is reconstructed using the minimum mean square error function.
[0048] Furthermore, after the pre-training is completed, the learning rate, network weights and biases of each layer of the model are fine-tuned through supervised overall training, including: first freezing the parameter update function of the pre-training stage, transferring the pre-trained weights and biases in each layer of the autoencoder to the corresponding layers of the deep neural network, and then conducting supervised training.
[0049] Compared with the prior art, the present invention has the following advantages:
[0050] The present invention introduces a layer-by-layer greedy pre-training structure in a deep neural network, and propagates the gradient between each layer by learning the features of each hidden layer, effectively alleviating the gradient vanishing problem, thereby improving the efficiency of the fitting model in the subsequent supervised fine-tuning stage. At the same time, the layer-by-layer greedy pre-training structure can accelerate the convergence speed of the deep neural network and improve the generalization ability of the fitting model. Therefore, it is feasible to combine the deep neural network with the layer-by-layer greedy pre-training structure to fit the propagation delay of the VHF signal in the marine propagation medium.
[0051] According to the relevant characteristics of marine meteorological information and the measured value of VHF signal propagation delay, the present invention introduces a layer-by-layer greedy pre-training structure into the deep neural network, and proposes a new method for accurately fitting the propagation delay of VHF signals in marine propagation media, so as to reduce the ranging error of VHF signals and improve the positioning accuracy of land-based navigation systems. BRIEF DESCRIPTION OF THE DRAWINGS
[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0053] Figure 1 The present invention is a flowchart of a VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure in an embodiment of the present invention.
[0054] Figure 2 4 is a diagram of the VHF signal propagation delay fitting model architecture in an embodiment of the present invention.
[0055] Figure 3 Schematic diagram of the layer-by-layer greedy pre-training structure in an embodiment of the present invention.
[0056] Figure 4 Schematic diagram of the autoencoder structure in an embodiment of the present invention.
[0057] Figure 5 Schematic diagram of the layer-by-layer pre-training process in an embodiment of the present invention. DETAILED DESCRIPTION
[0058] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0059] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0060] like Figure 1 As shown, the present invention discloses a VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure, comprising the following steps:
[0061] S1. Acquire historical VHF signal data, wherein the historical VHF signal data includes sea surface meteorological factor impact data and corresponding VHF signal propagation delay data.
[0062] Specifically, this step is used to collect neural network related training data. VHF signal propagation on the sea surface is susceptible to sea surface meteorological factors such as sea water temperature, humidity, air pressure, temperature, wind speed, wind direction and tidal water level. Therefore, in module 1, marine meteorological information such as sea water temperature, humidity, air pressure, temperature, wind speed, wind direction and tidal height will be extracted from the China Marine Forecast Network and the National Meteorological Science Data Center website, and the VHF signal propagation delay measured at the same time in the same sea area will be combined into a labeled training data set, 60% of the data in the data set will be used for subsequent training of the fitting model, 20% will be used as a validation set for model parameter adjustment, and 20% will be used as a test set for fitting model performance evaluation.
[0063] S2. Training a VHF signal propagation delay fitting model based on the historical VHF signal data. The VHF signal propagation delay fitting model adopts a four-layer deep neural network structure with multiple inputs and a single output and a dual hidden layer structure. The VHF signal propagation delay fitting model first adopts a layer-by-layer greedy training structure for unsupervised pre-training, and then fine-tunes the learning rate, network weights and bias of each layer of the model through supervised overall training. The layer-by-layer greedy training structure includes four autoencoder structures, which correspond to four levels of the VHF signal propagation delay fitting model respectively. Each autoencoder structure includes an encoder and a decoder. The encoder is used to adjust the dimension of the input sample data to the input dimension of the corresponding level of the VHF signal propagation delay fitting model, and the decoder is used to reconstruct the feature vector output by the encoder back to the dimension of the input sample data.
[0064] Taking into account the complexity and nonlinearity of the propagation delay fitting model, as well as the intrinsic correlation between the training data, this section uses a multi-input single-output four-layer deep neural network with a double hidden layer structure to construct a VHF signal propagation delay fitting model, wherein the first layer is the input layer, which uses GRU (gated recurrent unit) to extract the intrinsic correlation features between the input data, the second layer is the first hidden layer, and a multi-head attention mechanism is introduced to further extract data features, the third layer is the second hidden layer, which uses a fully connected structure to achieve multi-feature integration of data, and the fourth layer is the output layer, which uses a fully connected layer to output the VHF signal propagation delay fitting results. The network structure of the VHF signal propagation delay fitting model in the present invention is as follows: Figure 2 As shown. The specific idea is: the measured values of seawater temperature, humidity, air pressure, air temperature, wind speed, wind direction, tidal water level, and VHF signal propagation delay are used as training samples and input into the GRU layer. After training, the hidden state at each moment is obtained. All the hidden states obtained are combined into hidden state training and directly used as the input data of the second multi-head attention mechanism layer. The hidden state obtained in the first layer is linearly transformed into multiple query (Q), key (K) and value (V) vectors in the second layer for subsequent calculation of attention weights. A set of attention weight values is calculated for each head in the multi-head attention mechanism, and the weight values are used to weighted sum the query (Q), key (K) and value (V) vectors respectively. Then, the seven feature dimensions of the weighted sum of the query (Q), key (K) and value (V) are combined into a high-dimensional feature vector containing all the weighted summed query (Q), key (K) and value (V) vectors. The high-dimensional feature vector is used as the input data of the third fully connected layer. In this part, the ReLU activation function is used to transform the high-dimensional feature vector into a 32-dimensional feature vector. The 32-dimensional feature vector is used as the input data of the fourth fully connected layer. This part adopts the linear activation method to output the propagation delay fitting value of the VHF signal.
[0065] Specifically, due to the different measurement units of the input sample data, the scale ranges of the sample data features vary greatly. Input features of different scales will increase the difficulty of training, thereby affecting the overall performance of the fitting model. Therefore, this section normalizes the input sample data of the first hidden layer and the output layer to the same scale. The normalization method used is the layer normalization method. The steps of layer normalization are as follows:
[0066] Assume The net input of the neurons in the layer is , The mean and variance of are:
[0067] (1)
[0068] (2)
[0069] In the formula, is the activation vector length, which is the number of neurons in this layer. For the Layer neurons, Net input The mean of Net input The variance of . Thus, the net input of the layer normalization is for:
[0070] (3)
[0071] In the formula, is a numerical stability parameter, and its value is In order to prevent the normalized input sample data from having a negative impact on the expressive power of the deep neural network, it is necessary to add scaling and translation parameters to the normalized net input data. The scaling and translation parameters are and , so the net input data after scaling and translation is for:
[0072] (4)
[0073] The input layer of the present invention adopts the GRU layer to further analyze and extract the intrinsic characteristics of the input sample data to achieve the purpose of accurate fitting. The GRU layer used in this part first initializes the hidden state , and then for each group of vectors in the input sample matrix, the time step Do the following:
[0074] a. Calculate reset gate : Reset gate For the current Enter data at all times and the previous one Always hide status Perform a linear transformation to determine how the input data is combined with the previous hidden state. This step calculates the reset gate use Activation function, The calculation formula is:
[0075] (5)
[0076] In the formula, is the weight matrix, For the current Enter data at all times. For the previous Always hide the status, is the bias vector, is the Sigmoid activation function.
[0077] b. Calculate the update gate : Update gate For input data and the previous hidden state Perform linear changes to determine the degree to which the current time step retains the previous hidden state information. This step calculates the update gate use activation, The calculation formula is:
[0078] (6)
[0079] In the formula, is the weight matrix, For the current Enter data at all times. For the previous Always hide the status, is the bias vector, () is the Sigmoid activation function.
[0080] c. Calculate candidate hidden states : Candidate hidden state By resetting the gate and input Calculated. If the reset gate , then ignore the previous hidden state , Mainly depends on ; If the reset gate , most of them are retained Candidate hidden states The calculation adopts Activation function, The calculation formula is:
[0081] (7)
[0082] In the formula, is the weight matrix, for Reset the gate signal at all times. For the previous Always hide the status, is the bias vector, and tanh() represents the tanh activation function.
[0083] d. Calculate the current hidden state :According to the update gate With the previous Always hide status and candidate hidden states Determine the current Hidden state at the moment If the update gate ,but Close to ; If the update gate ,but Close to . The calculation formula is:
[0084] (8)
[0085] In the formula, for Update the door signal at all times. For the previous Always hide the status, is the candidate hidden state at the current moment.
[0086] In the second layer of the deep neural network, i.e. the first hidden layer, a multi-head attention mechanism is introduced. The hidden state sequence of all time steps in the GRU layer is directly used as the input data of this layer. The multi-head attention mechanism linearly transforms the input hidden state into multiple query (Q), key (K) and value (V) vectors, namely:
[0087] (9)
[0088] In the formula, , , is the learned weight matrix, Input all hidden states to the GRU layer. Formula (9) is used to calculate the single-head attention weight in the multi-head attention mechanism ,Right now:
[0089] (10)
[0090] In the formula, is the loss function, is the query matrix, is the transpose of the key matrix, is the value matrix, A set of attention weights is calculated by each attention mechanism head, and then the attention weights are used to weight the query (Q), key (K) and value (V) vectors respectively. Then, the vectors after the weighted sum of all single-head attention are combined in the seven feature dimensions of seawater temperature, humidity, air pressure, air temperature, wind speed, wind direction and tidal water level to form the final multi-head attention mechanism high-dimensional feature vector, namely:
[0091] (11)
[0092] In the formula, is the weighting function, The weighted sum of the query (Q), key (K), and value (V) vectors by the i-th attention head.
[0093] In the third layer of the deep neural network, i.e., the second hidden layer, a fully connected structure is used. The output data of formula (11) is used as the input data of this layer. The high-dimensional feature vector is then converted into a 32-dimensional feature vector through the fully connected structure. This layer uses the ReLU activation function. The output data of this layer is for:
[0094] (12)
[0095] In the formula, is the ReLU activation function, Enter data for this layer.
[0096] In the fourth layer of the deep neural network, i.e. the output layer of the fitting model, the fully connected structure is introduced again, and the propagation delay fitting value of the VHF signal in the sea surface propagation medium is obtained in a linear manner. ,Right now:
[0097] (13)
[0098] In the formula, is the output layer weight, is the output layer bias, The feature vector of the input of this layer.
[0099] In the present invention, in order to solve the problem that the deep neural network fitting model is prone to overfitting and leads to insufficient fitting accuracy, a greedy layer-by-layer pre-training structure is used to pre-train the neural network structure. The greedy layer-by-layer pre-training structure is composed of four autoencoders, each of which includes an encoder and a decoder. The encoder uses ReLU activation to adjust the dimension of the input sample data to the dimension of the corresponding layer of the deep neural network; the decoder uses Sigmoid activation to reconstruct the dimensional feature vector adjusted by the encoder back to the original input sample data dimension. The layer-by-layer greedy pre-training structure is trained in an unsupervised manner, and greedy parameter optimization and reconstruction are performed with the minimum loss function (MSE) as the cost function to obtain the optimized feature vector. After four layers of greedy layer-by-layer pre-training, the learning rate of the overall fitting model, the network weights and biases of each layer are fine-tuned by a supervised training method to achieve the overall fitting accuracy of the VHF signal propagation delay fitting of the present invention.
[0100] Specifically, since the hidden layer used in the present invention is not unique, the feature information extracted by the model is too complex, and the fitting model is prone to overfitting, resulting in poor fitting effect. To solve this problem, the present invention adds a layer-by-layer greedy pre-training structure to the deep neural network, such as Figure 3 As shown in the figure. The layer-by-layer greedy pre-training structure used in this section uses four autoencoders, each of which contains an encoder and a decoder. The encoder uses the ReLU activation function to adjust the dimension of the sample data to the dimension of the corresponding layer of the deep neural network; the decoder uses the Sigmoid activation function to reconstruct the dimension feature vector adjusted by the encoder back to the original input sample data dimension. The structure diagram of the autoencoder used in this section is shown in the figure. Figure 4 shown.
[0101] This part is divided into two stages: layer-by-layer pre-training stage and fine-tuning stage. The hierarchical representation of data features is learned through layer-by-layer training.
[0102] 1) Layer-by-layer pre-training stage
[0103] The pre-training process used in this section is as follows Figure 5 In the layer-by-layer pre-training stage, the training sample data in the dataset module is input into the first autoencoder in the form of feature vectors, and the encoder in the first autoencoder is trained in an unsupervised manner to obtain the output feature vectors. , The dimension of the feature vector output by the encoder is the same as that of the first layer of the deep neural network, the GRU layer. As the decoder input data in the first autoencoder, it is used to reconstruct the data dimension so that the output data dimension of the decoder is consistent with the original input data dimension, by minimizing the decoder input data and decoder output data The loss function is used to achieve the optimal reconstruction purpose. At the same time, the weights and biases of each pre-training are saved for subsequent fine-tuning of the entire deep neural network. The loss function used in this section is the minimum mean square error function, that is:
[0104] (14)
[0105] In the formula, is the number of input samples, For the reconstructed output samples, For the The original input samples.
[0106] At the same time, As the input data of the second autoencoder, the encoder in the second autoencoder is trained in an unsupervised manner to obtain the output features , The dimension of the feature vector output by the encoder of this layer is the same as that of the second layer of the deep neural network, that is, the first hidden layer. As the decoder input data in the second autoencoder, it is used to reconstruct the data dimension so that the output data dimension of the decoder is consistent with the input data dimension of the second encoder, and the output sample is reconstructed using the minimum mean square error function. As the input data of the third autoencoder, the above training process is repeated until the fourth autoencoder training is completed.
[0107] After the pre-training stage, it was determined that the GRU layer of the deep neural network uses 64 neuron nodes, the number of attention heads in the first hidden layer is 32, the number of hidden nodes in the second hidden layer is 32, and the number of hidden nodes in the output layer is 16.
[0108] 2) Fine-tuning stage
[0109] In the fine-tuning stage, the parameter update function of the pre-training stage is first frozen, and the pre-trained weights and biases in each layer of the autoencoder are transferred to the corresponding layers of the deep neural network, and then supervised training is performed. The mean absolute error is used to evaluate the model fitting performance by monitoring the convergence of the validation set and the fitting effect of the test set. , Root Mean Square Error and the coefficient of determination .
[0110] The expression is:
[0111] (15)
[0112] In the formula, is the number of data, is the measured value of VHF signal propagation delay, is the fitting value of the VHF signal propagation delay of the fitting model
[0113] The expression is:
[0114] (16)
[0115] In the formula, is the number of data, is the measured value of VHF signal propagation delay, is the fitting value of the VHF signal propagation delay of the fitting model
[0116] The expression is:
[0117] (17)
[0118] In the formula, is the number of data, is the measured value of VHF signal propagation delay, To fit the model to the propagation delay of VHF signal, is the average of the measured values of the VHF signal propagation delay. In formulas (15) to (17), The smaller it is, the more accurate the fitting result of the fitting model is; The smaller it is, the better the fitting performance of the fitting model is; The value range is , The closer it is to 1, the better the fitting effect of the fitting model is, that is, the smaller the difference between the propagation delay fitting value of the VHF signal and the true value is.
[0119] When the fitting effect of the test set does not meet expectations, the parameter update function of the pre-training stage is unfrozen and the labeled data is retrained. By adjusting the learning rate, the convergence of the validation set and the fitting effect of the test set are monitored.
[0120] This section uses the Adam optimizer to adjust the learning rate. The current moment is used in the Adam optimizer Dynamic variables and exponentially weighted average variables , and The expressions are:
[0121] (18)
[0122] (19)
[0123] In the formula, For the current moment The gradient of For the previous Dynamic variables at any time, For the previous The exponentially weighted average variable at time, and is the hyperparameter momentum decay rate, which is set to 0.9 in the present invention. When dynamic variables and exponentially weighted average variables The initial value of is 0, resulting in and The value is lower than that at other times. Therefore, this section corrects the deviation of the dynamic variables and exponentially weighted average variables in the Adam optimizer. The corrected dynamic variables and exponentially weighted average variables They are:
[0124] (20)
[0125] (twenty one)
[0126] In the formula, and The momentum decay rates are and of Power, is the current moment. After each round of training, the current moment The weight and bias parameter set To update, that is:
[0127] (twenty two)
[0128] In the formula, for The weight and bias parameter set at the moment, is the learning rate, is a numerical stability parameter, usually set to 10 -8 .
[0129] S3. Obtain the current sea surface meteorological factor impact data, output the trained VHF signal propagation delay fitting model, and obtain the model output as the VHF signal propagation delay data fitting result.
[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure, characterized in that: The following steps are involved: Acquiring historical VHF signal data, wherein the historical VHF signal data includes sea surface meteorological factor impact data and corresponding VHF signal propagation delay data; A VHF signal propagation delay fitting model is trained based on the historical VHF signal data. The VHF signal propagation delay fitting model adopts a four-layer deep neural network structure with multiple inputs and a single output and a dual hidden layer structure. The VHF signal propagation delay fitting model first adopts a layer-by-layer greedy training structure for unsupervised pre-training, and then fine-tunes the learning rate, network weights and bias of each layer of the model through supervised overall training. The layer-by-layer greedy training structure includes four autoencoder structures, and the four autoencoder structures correspond to four levels of the VHF signal propagation delay fitting model, respectively. Each autoencoder structure includes an encoder and a decoder. The encoder is used to adjust the dimension of the input sample data to the input dimension of the corresponding level of the VHF signal propagation delay fitting model, and the decoder is used to restore the feature vector output by the encoder to the dimension of the input sample data. The influence data of the current sea surface meteorological factors are obtained, the trained VHF signal propagation delay fitting model is output, and the model output is obtained as the VHF signal propagation delay data fitting result.
2. A VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure according to claim 1, characterized in that: The sea surface meteorological factors influencing data include sea water temperature, humidity, air pressure, air temperature, wind speed, wind direction and tidal water level.
3. A VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure according to claim 1, characterized in that: The VHF signal propagation delay fitting model comprises an input layer, a first hidden layer, a second hidden layer and an output layer, wherein: The input layer uses a gated recurrent unit to extract intrinsic correlation features between input data; The first hidden layer introduces a multi-head attention mechanism to further extract data features; The second hidden layer is a fully connected layer for realizing multi-feature integration of data; The output layer adopts a fully connected layer to output the VHF signal propagation delay fitting result.
4. A VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure according to claim 3, characterized in that: The input layer extracts the intrinsic related features between the input data, including: First initialize the hidden state , and then for each group of vectors in the input sample matrix, the time step Do the following: a. Calculate reset gate : Calculate reset gate use Activation function, The calculation formula is: In the formula, is the weight matrix, For the current Enter data at all times. For the previous Always hide the status, is the bias vector, is the Sigmoid activation function; b. Calculate the update gate :Calculate the update gate use activation, The calculation formula is: In the formula, is the weight matrix, For the current Enter data at all times. For the previous Always hide the status, is the bias vector, is the Sigmoid activation function; c. Calculate candidate hidden states : Candidate hidden state The calculation adopts Activation function, The calculation formula is: In the formula, is the weight matrix, for Reset the gate signal at all times. For the previous Always hide the status, is the bias vector; d. Calculate the current hidden state : The calculation formula is: In the formula, for Update the door signal at all times. For the previous Always hide the status, is the candidate hidden state at the current moment.
5. A VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure according to claim 3, characterized in that: The data features are further extracted through the first hidden layer, including: The hidden state obtained from the input layer is transformed into a query vector, a key vector and a value vector through a linear transformation; a set of attention weight values is calculated for each head in the multi-head attention mechanism, and the weight values are used to perform weighted summation on the query vector, key vector and value vector respectively; the weighted summed query vector, key vector and value vector are combined into a high-dimensional feature vector containing all the weighted summed query vectors, key vectors and value vectors.
6. A VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure according to claim 3, characterized in that: The input sample data of the first hidden layer and the output layer are normalized to the same scale, and the normalization steps are as follows: Assume The net input of the neurons in the layer is , The mean and variance of are: In the formula, is the activation vector length, which is the number of neurons in this layer. For the Layer neurons, Net input The mean of Net input The variance of , from which we can get the net input of layer normalization for: In the formula, is a numerical stability parameter, and its value is ; Add scaling and translation parameters to the normalized net input data. for: in, is the scaling parameter, is the translation parameter.
7. A VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure according to claim 1, characterized in that: The VHF signal propagation delay fitting model adopts a layer-by-layer greedy training structure for unsupervised pre-training, including: The training sample data is input into the first autoencoder in the form of feature vectors, and the encoder in the first autoencoder structure is trained in an unsupervised manner to obtain the output feature vector. , For the encoder output feature vector dimension in the first autoencoder structure, As the decoder input data in the first autoencoder, it is used to reconstruct the data dimension so that the output data dimension of the decoder is consistent with the original input data dimension, by minimizing the decoder input data and decoder output data The loss function is used to achieve the optimal reconstruction purpose, while preserving the weights and biases of each pre-training; Will As the input data of the second autoencoder, the encoder in the second autoencoder is trained in an unsupervised manner to obtain the output features , For the encoder output feature vector dimension in the second autoencoder structure, As the decoder input data in the second autoencoder, it is used to reconstruct the data dimension so that the output data dimension of the decoder is consistent with the input data dimension of the second encoder, and the output sample is reconstructed using the minimum mean square error function; Will As the input data of the third autoencoder, the encoder in the third autoencoder is trained in an unsupervised manner to obtain the output features , o is the dimension of the encoder output feature vector in the third self-encoder structure, As the decoder input data in the third autoencoder, it is used to reconstruct the data dimension so that the output data dimension of the decoder is consistent with the input data dimension of the third encoder, and the output sample is reconstructed using the minimum mean square error function; Will As the input data of the fourth autoencoder, the encoder in the fourth autoencoder is trained in an unsupervised manner to obtain the output feature , p For the encoder output feature vector dimension in the fourth autoencoder structure, As the decoder input data in the fourth autoencoder, it is used to reconstruct the data dimension so that the output data dimension of the decoder is consistent with the input data dimension of the fourth encoder, and the output sample is reconstructed using the minimum mean square error function.
8. A VHF signal propagation delay fitting method based on a layer-by-layer greedy pre-training structure according to claim 7, characterized in that: After the pre-training, the model's learning rate, network weights and biases of each layer are fine-tuned through supervised overall training, including: first freezing the parameter update function of the pre-training stage, transferring the pre-trained weights and biases in each layer of the autoencoder to the corresponding layers of the deep neural network, and then conducting supervised training.
Citation Information
Patent Citations
Speech recognition method adopting rectified deep auto-encoder network
CN106782511A
VHF signal propagation path error prediction method and system
CN114742265A
Multi-task ship navigation dynamic perception and identification method based on pre-training model
CN116188984A
Spectral detection and localization of radio events with learned convolutional neural features
US11630996B1
Anomaly Detection and Diagnosis in Factory Automation System using Pre-Processed Time-Delay Neural Network with Loss Function Adaptation
US20230068908A1
