Anomaly detection driven sea ice semi-supervised multi-task inversion method

Abstract

The application discloses an abnormality detection driven sea ice semi-supervised multi-task inversion method and belongs to the technical field of satellite remote sensing and deep learning. The application solves the problems that the existing method excessively depends on labels, is structurally redundant and cannot effectively invert sea ice under the condition of sample imbalance. The method comprises the following steps: inputting DDM data of TDS-1 into a data preprocessing module to obtain standard data required by a multi-task network; using a sea ice detection module in the multi-task network structure to perform deep feature extraction on the input standard DDM data, using a latent deep learning guide mechanism to mine and share detailed information and spatial context, accurately distinguishing sea ice data and seawater data through the module to complete a sea ice detection task; then inputting the extracted DDM high-dimensional features and the module output results into a sea ice concentration inversion module through an inter-module data feature sharing mechanism, using the learned features to infer the concentration of sea ice and completing sea ice concentration inversion. The application effectively improves the precision and efficiency of sea ice inversion.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of satellite remote sensing and deep learning, and particularly relates to an abnormality detection driven sea ice semi-supervised multi-task inversion method. BACKGROUND

[0002] The formation, melting and movement of sea ice have a significant impact on the stability of the earth's climate system, the ecological balance of the ocean and human social and economic activities. Therefore, it has high professional value and practical significance to carry out accurate detection and analysis of sea ice.

[0003] At present, sea ice detection using satellite remote sensing is mainly based on visible light remote sensing and microwave remote sensing. The images obtained by visible light remote sensing are affected by weather conditions, and cannot be observed all-weather and in real time. Microwave remote sensing is not affected by weather, but its spatial resolution is poor, and increasing the antenna size is needed to improve the spatial resolution, which is a challenge under the current space technology conditions.

[0004] With the development of global navigation satellite system, global navigation satellite system reflectometry (GNSS-R) technology has become a popular means of global sea ice detection as a new method. The existing GNSS-R based sea ice remote sensing methods mainly include methods based on Delay Doppler Maps (DDM) observable values, methods based on scatter coefficient retrieval and methods based on machine learning. Although the methods based on DDM observable values and the methods based on scatter coefficient retrieval can realize the sea ice detection task, both of these two methods have certain limitations, and cannot directly learn information from DDM data, and need to rely on designed rules or feature extraction for sea ice detection, while machine learning can automatically learn the relationship between image and required parameters from a large amount of remote sensing image data without specific rules; on the other hand, the above two methods only complete the sea ice detection task, and do not perform sea ice concentration inversion task. Therefore, the method of using machine learning for sea ice remote sensing is gradually explored by many scholars, which not only improves the performance of sea ice detection, but also most of the machine learning algorithms also design similar networks to complete the task of sea ice concentration inversion.

[0005] Although these machine learning-based methods have made significant progress in advancing sea ice retrieval technology, they are mainly supervised learning algorithms, which means they are highly dependent on accurate and high-quality labeled data. In practical operation, it is not only challenging but also costly to obtain a large-scale, high-quality labeled dataset. More importantly, these labeled data are likely to be affected by various errors, which will affect the accuracy of the final results. Therefore, although these methods are theoretically very valuable, their flexibility and applicability in practical applications still have certain limitations. In addition, for specific situations in practical application scenarios, although we can obtain a large amount of sea ice DDM data, the amount of sea water DDM data is much larger than that of sea ice DDM data on a global scale. This phenomenon leads to the problem of sample imbalance when using a large-scale dataset for model training. How to overcome the two challenges of excessive dependence on labeled data and sample imbalance is of great significance in this field. SUMMARY

[0006] The purpose of the present application is to provide an anomaly detection driven sea ice semi-supervised multi-task retrieval method to solve the problem of excessive dependence on labels, structural redundancy and ineffective retrieval of sea ice in the case of sample imbalance in the prior art.

[0007] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0008] An anomaly detection driven sea ice semi-supervised multi-task retrieval method, comprising the following steps:

[0009] S1, obtaining DDM data of TDS-1, inputting the obtained DDM data into a data preprocessing module, and obtaining standard DDM data that can be used for sea ice multi-task retrieval through data screening, background noise subtraction and normalization;

[0010] S2, designing a multi-task grid structure, the multi-task grid structure comprising a sea ice detection module and a sea ice concentration retrieval module; inputting sea water DDM data in the standard DDM data obtained in S1 into the sea ice detection module in the multi-task grid structure; performing deep feature extraction on the input sea water DDM data to accurately distinguish sea ice data and sea water data, and completing the sea ice detection task;

[0011] S3, inputting the high-dimensional features of the sea water DDM data extracted by the sea ice detection module and the module output results into the sea ice concentration retrieval module through a latent deep learning guidance mechanism, inferring the concentration of sea ice using the learned features, and completing the sea ice concentration retrieval task.

[0012] Preferably, the screening condition of the data screening work in S1 is:

[0013] ① Data with latitudes greater than 55 degrees north and south;

[0014] ② Data with satellite incidence angle less than 35 degrees;

[0015] ③ Peak signal-to-noise ratio (SNR) greater than 0dB;

[0016] ④Antenna gain is greater than 3dB;

[0017] ⑤The sea ice concentration is 0.15 as the dividing line.

[0018] Preferably, the background noise subtraction in S1 specifically includes the following:

[0019] The calculation formula for the noise floor is:

[0020]

[0021] Based on formula (1), the formula for subtracting the background noise of each DDM data is:

[0022] DDM remove_noise =DDM(τ,f)-Noise (2)

[0023] Where Noise represents the noise value; τ1 and τ2 represent the pixel limits within the delay frequency band; f1 and f2 represent the pixel limits within the Doppler frequency band; X represents the total number of pixels within the noise box; and DDM(τ, f) represents the value at a specific position in the original DDM data in the delay and Doppler frequency directions.

[0024] Preferably, the normalization operation in S1 specifically refers to normalizing each DDM according to its peak power, and the normalization calculation formula is:

[0025]

[0026] Among them, DDM(τ,f) represents the value of a specific position of the original DDM data in the direction of time delay and Doppler frequency, DDM maxnosie The DDM value represents the peak power after subtracting the noise floor.

[0027] Preferably, the sea ice detection module is used for anomaly detection and includes a generator G(x) and a discriminator D(x);

[0028] The DDM image reflected by seawater is input into the sea ice detection module as a normal sample for training, and the sea ice DDM image is used as an abnormal sample. During the training process, the generator G(x) and the discriminator D(x) of the sea ice detection module compete with each other, wherein the generator G(x) is used to generate a reconstructed DDM image that is closer to the real DDM image x. so that the discriminator D(x) cannot distinguish between x and

[0029] Preferably, the generator G(x) is composed of an encoder (Encoder) and a decoder (Decoder); the encoder (Encoder) maps the input image to a latent vector in the latent space by downsampling, extracting abstract feature representation; the encoder (Encoder) contains convolutional layers and BatchNorm layers and LeakyReLU activation functions at each layer, reads the input data into the network, and outputs the latent vector in the latent space;

[0030] The decoder (Decoder) maps the latent vector in the latent space back to the original image space by upsampling, i.e. reconstructs the abstract feature representation into an image again, and finally generates a realistic image;

[0031] The decoder (Decoder) adopts the structure of transposed convolution Transposed Convolution and BatchNorm layer, which corresponds one-to-one with the structure of the encoder (Encoder); the transposed convolution is used to upsample the input latent space vector and increase the feature map size; the decoder (Decoder) uses the Tanh activation function in the output layer, and the ReLU activation function in other layers.

[0032] Preferably, the D(x) is composed of convolutional layers and BatchNorm layers and LeakyReLU activation functions; the discriminator D(x) is used to judge the true or false of the reconstructed image generated by the generator G(x) and the input real image, and to classify them, and the goal of the discriminator D(x) is to get a high score on the real image and a low score on the reconstructed image;

[0033] The discriminator D(x) also serves as a feature extractor, outputs the intermediate layer of the discriminator as a feature, and then calculates the feature matching loss between the generated image and the real image.

[0034] Preferably, the latent deep learning guided mechanism in S3 is implemented by a skip connection, which makes the information jump in the network, connects the feature map of the last layer in the encoder (Encoder) with the feature map of the corresponding layer in the decoder (Decoder) to pass more detailed information and spatial context, and restores more rich detail information in the decoder stage.

[0035] Preferably, the sea ice concentration inversion module in S3 includes a discriminant matching unit and an LSTM (Long Short-Term Memory) network.

[0036] The discrimination matching unit judges and matches the output result of the sea ice detection module with the original input image, and data judged as abnormal by the sea ice detection module is taken as input of the LSTM network; the LSTM network receives the output of the matching discrimination unit, and obtains high-dimensional feature representation of corresponding DDM data through an inter-module data feature sharing mechanism, thereby completing the sea ice concentration inversion task.

[0037] The LSTM network comprises two stacked LSTM layers, each of which is regarded as a layer performing sequence modeling on input data, and the stacking of multiple layers allows the model to capture more complex sequence dependencies; each LSTM layer comprises 128 hidden units, each of which comprises a memory cell; three key gating mechanisms, including a forget gate, an input gate and an output gate, are included in the LSTM to control the flow and update of cell states.

[0038] Preferably, the forget gate is used to determine how much cell state of the previous time is retained to the cell state of the current time, and the specific function is represented as:

[0039] f t =σ(W f ·[h t-1 ,x t ]+b f ) (4)

[0040] Wherein, f t represents the output value of the forget gate; sigma represents a sigmoid layer; w f represents a weight matrix of the forget gate; x t represents input of the current time step; h t-1 represents hidden state of the previous time step; [h t-1 , x t ] represents a vector formed by connecting the hidden state of the previous time step and the input of the current time step; and b f represents a bias vector.

[0041] The input gate decides which information should enter the cell state through learned weights, and comprises a sigmoid layer and a tanh layer; the sigmoid layer decides what value to update, and the tanh layer updates the information to be updated to the cell state, and the specific function is represented as:

[0042] i t =σ(W i ·[h t -1,x t ])+bi (5)

[0043]

[0044] Wherein, i t represents the output value of the input gate; represents the candidate value of the input gate;W i , W f represents the weight matrix;b i , b c represents the bias vector;

[0045] The memory in the new cell state C t is composed of two parts, the old memory left after the useless memory of the memory cell at the last time is forgotten and the useful information filtered out as new memory in the input gate;The function representation of the new cell state is as follows:

[0046]

[0047] The output gate is used to determine which information should be output to the next layer or as the final output, and the specific function representation is as follows:

[0048] o t = sigma (W o [ h t-1 , x t ] + b o ) (8)

[0049] h t = ot * tanh (C t ) (9)

[0050] Wherein, o t represents the output value of the output gate;h t represents the hidden state of the current time step;

[0051] The hidden state is part of the LSTM network output, used to carry the information of the network at the current time step;The hidden state is useful information extracted from the cell state, used for network prediction or input of the next time step.

[0052] Compared with the prior art, the present application provides an abnormal detection driven sea ice semi-supervised multi-task inversion method, which has the following beneficial effects:

[0053] (1) In order to solve the problem of huge data labeling workload, and overcome the problem of imbalance between sea ice and seawater DDM data, the present application first applies the semi-supervised anomaly detection idea to the field of remote sensing sea ice inversion, and realizes the sea ice inversion task under limited data label.

[0054] (2) In order to optimize the algorithm structure of sea ice retrieval and reduce its redundancy, the present application proposes an innovative integrated sea ice retrieval framework. This multi-task network structure uniquely integrates the sea ice detection and sea ice concentration retrieval modules, achieving both tasks through a single network. In the sea ice detection module, the present application uses a latent deep learning guidance mechanism to mine and share detailed information and spatial context. At the same time, in the sea ice concentration retrieval module, through an inter-module data feature sharing mechanism, the DDM data features of the sea ice detection module are shared to the time series feature extraction network, reducing the data feature repeated extraction process and significantly reducing the model structure and parameter redundancy.

[0055] (3) The present application not only focuses on spatial texture features, but also considers the time series relationship of different features. While extracting DDM texture features, the data regression strategy is further optimized according to the time series relationship between features. Through high-dimensional texture feature construction and special attention to time series features, the algorithm can more effectively capture the local details of the data and its relationship with global features, thereby effectively enhancing the data understanding ability and generalization ability of the model in complex data environments. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 is the overall structure diagram of the abnormality detection driven sea ice semi-supervised multi-task retrieval method mentioned in embodiment 1 of the present application;

[0057] Figure 2 is a specific structure diagram of the multi-task network mentioned in embodiment 1 of the present application;

[0058] Figure 3 is a DDM image diagram mentioned in embodiment 1 of the present application; wherein, Figure 3 (a) is a seawater image, Figure 3 (b) is an image with sea ice concentration of 50%, Figure 3 (c) is an image with sea ice concentration of 94%. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical scheme and advantages of the present application clearer, the following further describes the embodiments of the present application in detail.

[0060] Embodiment 1:

[0061] Please refer to Figure 1 , the present application proposes a hybrid model sea ice range prediction algorithm based on feature key capture mechanism, as Figure 1 shown, comprising the following steps:

[0062] Step 1, obtain the DDM data of TDS-1, input the DDM image into the data preprocessing module, through data screening, background noise subtraction and standardization, obtain the standard DDM data which can be used for sea ice multi-task inversion.

[0063] TDS-1 is a satellite launched in 2014 for spaceborne GNSS-R research. Its operational orbit has an inclination of 98.4° and an altitude of 635 km. The satellite carries eight independent payloads. One of them is the Space Global Navigation Satellite System Receiver Remote Sensing Instrument (SGR-ReSI), which generates DDM data by cross-correlating the scattered signal with locally generated code replicas with different time delays and Doppler shifts.

[0064] DDM is a scattering power map composed of 128 delay bins, each with a resolution of 244 ns, and 20 Doppler bins, each with a resolution of 500 Hz. DDM is affected by key parameters such as the location of the specular reflection point (SP), the incidence angle of the satellite, the gain of the antenna, and the time of data acquisition. In this study, we use the SP location provided in the DDM metadata as the acquisition location of the DDM. SP can be used to extract SIC corresponding to sea ice DDM from reference data.

[0065] As shown in Figure 3 , respectively, are the DDM images of seawater, sea ice concentration of 50%, and sea ice concentration of 94%. The SIC value is obtained from the reference data corresponding to the SP location of each DDM. As can be seen from Figure 3 , for different states of the ocean surface (seawater, sea ice, and different sea ice concentrations), DDM shows different image characteristics. Therefore, by analyzing the characteristics of the DDM graph, sea ice and seawater, as well as different sea ice distribution states, can be effectively distinguished. This is because the smoothness of the reflecting surface plays a key role in the intensity of coherent reflection. Specifically, the smoother the surface, the stronger the coherent reflection, and the smaller the degree of diffusion of the DDM flicker area. Conversely, the greater the degree of diffusion. Since the surface of sea ice is relatively smooth, the scattering power of sea ice surface is almost entirely from SP, so the DDM graph shows a more pronounced peak and a smaller flicker area, as shown in Figure 3 (c). On the contrary, the DDM graph of seawater usually exhibits a larger diffusion area and presents a horseshoe shape, as shown in Figure 3 (a).

[0066] According to the ratio of seawater and sea ice DDM obtained by TDS-1 in 2018, sea ice DDM data is 8.8% of seawater DDM data, and sea ice DDM has different forms according to different concentrations, so in this invention, seawater DDM is regarded as normal data for training, and sea ice DDM is regarded as abnormal data

[0067] The data set preparation process in this embodiment is:

[0068] (1) Data screening. Since the acquired DDM is not all available for sea ice inversion, quality control is needed according to certain conditions. The data set used in this application is mainly screened according to the following conditions:

[0069] The north and south latitudes are greater than 55 degrees. Since sea ice data generally exists in high latitude areas, DDM data is selected from the area with latitude above 55 degrees in the south and north latitude areas as the sea ice data in the SND2018 data set.

[0070] The satellite incidence angle is less than 35 degrees. The smaller the satellite incidence angle, the more directly the signal is received by the satellite, and the smaller the path loss. Therefore, the smaller the incidence angle, the smaller the coherent scattering component, and the better the signal transmission quality.

[0071] The peak signal-to-noise ratio (SNR) is greater than 0 dB. The signal-to-noise ratio is an index for measuring the relationship between signal and noise, and is used to describe the relationship between the strength of the useful signal and the noise level. A lower SNR indicates that the DDM is affected by noise and cannot correctly display the characteristics of the reflecting surface. Therefore, data with SNR greater than 0 dB is selected to remove all noise points below the threshold.

[0072] The antenna gain is greater than 3 dB. The antenna gain is the efficiency of the antenna in receiving or transmitting signals in a certain direction. Therefore, the higher the antenna gain, the higher the efficiency of the antenna in receiving or transmitting signals in a certain direction, representing better communication quality.

[0073] The sea ice concentration 0.15 is the dividing point. Data with SIC value higher than 15% is considered to contain sea ice reflection signal, otherwise it is considered to be sea water reflection signal data.

[0074] (2) Data background noise subtraction. In order to further improve the data quality and reduce the influence of noise generated by the antenna and instrument on the data, the method of background noise subtraction is used for processing. The background noise is the pixel frame without signal, specifically the first four delay frequency bands of all Doppler frequency bands contained in each DDM. The noise calculation formula is shown in formula (1).

[0075]

[0076] Wherein, τ1, τ2 represent the pixel limit in the time delay frequency band; f1, f2 represent the pixel limit in the Doppler frequency band; X represents the number of all pixels in the noise frame; DDM(τ, f) represents the value of a certain specific position of the original DDM data in the time delay and Doppler frequency directions; Noise represents the noise value. After calculating the noise, subtract the noise in each bin of the DDM, as shown in formula 2.

[0077] DDM remove_noise = DDM (τ, f) - Noise (2)

[0078] (3) Data Standardization

[0079] Since the DDM data acquired by TDS-1 is not calibrated, it is necessary to normalize each DDM according to its peak power after removing the background noise. The normalization formula is shown in formula 3.

[0080]

[0081] Wherein, DDM (τ, f) represents the value of a certain position of the original DDM data in the time delay and Doppler frequency direction, DDM maxnosie represents the DDM value after subtracting the peak power of the background noise.

[0082] Step 2, input the obtained seawater DDM data in the standard DDM data into the sea ice detection module in the multi-task network structure, accurately distinguish the sea ice data and the seawater data by deep feature extraction on the input DDM data, and complete the sea ice detection task.

[0083] The specific structure diagram of the multi-task network proposed in the application is shown in Figure 2 . The network includes a sea ice detection module and a sea ice concentration inversion module.

[0084] The main idea of the sea ice detection module is anomaly detection. Since the sea ice DDM data only accounts for 8% of the seawater DDM data, the application takes the DDM image reflected by seawater as a normal sample and the sea ice DDM image as an abnormal sample. The module takes Deep Convolutional Generative Adversarial Networks (DCGAN) as the main architecture for feature learning and reconstruction of normal samples. It is composed of a generator G (x) and a discriminator D (x). G (x) and D (x) play against each other during the training process. The purpose of G (x) is to generate a reconstructed DDM image that is more similar to the real DDM image x, so that D (x) cannot distinguish true and false, while the purpose of D (x) is to perfectly distinguish x and . The final result of this game is that x and are more and more indistinguishable, and finally make G (x) generate a realistic , which means that the network has learned the features of normal seawater data. Then, in the subsequent test, if the input is sea ice data (that is, abnormal data), since the generator has not learned the features of sea ice data, G (x) generates There will be a big difference with the input sea ice data, so D(x) can easily determine the difference between the input image and the reconstructed image, so as to achieve the purpose of sea ice detection.

[0085] The generator is composed of an encoder and a decoder. The encoder maps the input image to the latent vector in the latent space through downsampling, and extracts the abstract feature representation. The encoder contains a convolution layer, a BatchNorm layer and a LeakyReLU activation function at each layer. The four layers read the input data into the network and output the latent vector in the latent space. The feature maps output by each layer of the encoder are (10, 64, 64), (5, 32, 128) and (2, 9, 256) respectively. The decoder maps the latent vector in the latent space back to the original image space through upsampling, i.e. reconstructs the abstract feature representation into an image again. The final decoder generates a realistic image, which has similar features to the original input image. The structure of the decoder corresponds to that of the encoder, but the decoder uses the structure of Transposed Convolution and BatchNorm layer. Transposed Convolution is used to upsample the input latent space vector and increase the feature map size. Since the sea ice DDM data used in the present application is rectangular, in order to be able to reconstruct a (20, 128) image, the convolution kernel, step size and padding of some layers also need to be adjusted accordingly, such as the convolution kernel of the first layer of Transposed Convolution is 4, the step size is (3, 4), and the padding is (1, 2). In addition, the decoder uses the Tanh activation function in the output layer, and the ReLU activation function is used in other layers. The feature maps output by each layer of the decoder are (2, 9, 256), (5, 32, 128) and (10, 64, 64), and finally the reconstructed image (20, 128, 1) is output.

[0086] D(x) is used to determine whether the reconstructed image generated by G(x) is true or false, and to classify them. The goal of D(x) is to get a high score on the real image and a low score on the reconstructed image. In addition to the above functions, the D(x) of the SIMTRN proposed in the present application also serves as a feature extractor, which outputs the intermediate layer of the discriminator as a feature, and then calculates the feature matching loss between the generated image and the real image. This helps to ensure that the generated image is similar to the real image in the feature space. D(x) is composed of a convolution layer, a BatchNorm layer and a LeakyReLU activation function. The output of each layer is (20, 128, 64), (10, 64, 128), (5, 32, 256), (2, 9, 512) and (2, 8, 100).

[0087] Step 3, the high-dimensional features of the seawater DDM extracted by the sea ice detection module and the module output result are input to the sea ice concentration inversion module through a latent deep learning guiding mechanism, and the learned features are used to infer the concentration of sea ice, thereby completing the sea ice concentration inversion task.

[0088] The latent deep learning guiding mechanism is realized through a skip connection. The mechanism allows information to be transmitted in the network by jumping, connects the feature map of the last layer in the encoder with the feature map of the corresponding layer in the decoder, helps to transmit more detailed information and spatial context, restores more detailed information in the decoder stage, especially when processing edge and local features, maximizes the retention and restoration of detail loss in the indirect propagation process. This also helps to alleviate the gradient vanishing problem, makes the network easier to train, reduces the reconstruction difficulty, generates higher quality images, and thus improves the performance of the entire network.

[0089] The sea ice concentration inversion module includes a discriminant matching unit and an LSTM (Long Short-Term Memory) network. The discriminant matching unit judges and matches the output result of the sea ice detection module with the original input image, and data that is sea ice data and is judged as abnormal by the sea ice detection module is input to the LSTM network. The LSTM network receives the output of the matching and discrimination unit, and obtains the high-dimensional feature representation of the corresponding DDM data through an inter-module data feature sharing mechanism, thereby completing the sea ice concentration inversion task.

[0090] The LSTM used in the application includes two stacked LSTM layers, each of which can be regarded as a layer that performs sequence modeling on input data, and the stacking of multiple layers allows the model to capture more complex sequence dependencies. Each LSTM layer has 128 hidden units, and each hidden unit includes a memory unit. The memory unit consists of a cell state and a hidden state. Both of them are information transmitted in the network.

[0091] Specifically, the cell state is the main memory unit of the LSTM network, responsible for saving and transmitting information. t The state of the unit at time t. At each time step, the cell state is updated, and some information can be selectively retained or deleted. In LSTM, there are three key gating mechanisms to control the flow and update of the cell state: the forget gate, the input gate, and the output gate.

[0092] The forget gate determines how much of the unit state at the previous time is retained in the unit state at the current time, that is, which information should be forgotten. t is the output value of the forget gate, as shown in equation 4.

[0093] f t = σ(W f · [h t-1 , x t ] + b f ) (4)

[0094] where f t represents the output value of the forget gate; σ represents the sigmoid layer; w f represents the weight matrix of the forget gate; x t represents the input of the current time step; h t-1 represents the hidden state of the previous time step; [h t-1 , x t ] represents a vector formed by connecting the hidden state of the previous time step and the input of the current time step; and b f represents the bias vector.

[0095] The input gate decides which information should enter the cell state through learned weights. It contains two parts. The first part is the sigmoid layer, which decides what values to update. The second part is the tanh layer, which updates the information that needs to be updated to the cell state.i t is the output value of the input gate.i t and as shown in equations 5 and 6.

[0096] i t = σ(W i · [h t -1, x t ] + b i ) (5)

[0097]

[0098] where i t represents the output value of the input gate; and represents the candidate value of the input gate; W i , W f represent the weight matrix; and b i , b c represent the bias vector.

[0099] The memory in the new cell state C t is composed of two parts. One part is the old memory left after the memory cell forgets the useless memory of the last time. The other part is the useful information selected as new memory in the input gate. The formula of the new cell state is shown in equation 7.

[0100]

[0101] The output gate determines which information should be output to the next layer or as the final output. t is the output value of the output gate. t is the output value of the output gate. t As shown in equations 8 and 9.

[0102] o t = σ(W o [h t-1 , x t ]+b o ) (8)

[0103] h t = ot*tanh(C t ) (9)

[0104] wherein o t represents the output value of the output gate; h t represents the hidden state of the current time step;

[0105] The hidden state is part of the LSTM network output, used to carry the information of the network at the current time step. The hidden state can be considered as useful information extracted from the cell state, used for the prediction of the network or the input of the next time step.

[0106] In the sea ice detection module, the ultimate goal of the generator and the discriminator playing against each other is to make the generator generate more and more realistic reconstructed images, i.e., the generator learns the characteristics of the normal data. In order to achieve the purpose, a loss function needs to be used to optimize the model. The sea ice detection module proposed in the present application uses three kinds of loss functions:

[0107] (1) Adversarial loss:

[0108] The adversarial loss is the game loss between the generator and the discriminator. Its goal is to make the reconstructed image of the generator more and more realistic. The goal of the generator is to minimize the probability that the reconstructed image is judged as false by the discriminator as much as possible, and the goal of the discriminator is to maximize the probability of correctly distinguishing the real image and the reconstructed image as much as possible. The expression of the adversarial loss is shown in equation 10:

[0109]

[0110] wherein, is the expectation for real data, which represents the expectation of the probability that the discriminator judges the real data as real; is the expectation for reconstructed data, which represents the expectation of the probability that the discriminator judges the generated data as false; x ~ p x represents sampling from the real data distribution.

[0111] (2) Context loss:

[0112] The adversarial loss can ensure that the reconstructed image of the generator is increasingly realistic, but it cannot guarantee that the model can learn the context of the input. The context loss is used to measure the difference in structure and context information between the generated image and the real image. The goal is to ensure that the generated image is similar to the real image at the pixel level, while considering their contextual relationship. This loss can ensure that the model generates images similar to the context of normal samples, rather than generating reconstructed images that deviate from the structure of real images. The expression of the context loss is shown in Equation 11:

[0113]

[0114] wherein, represents the L1 loss, that is, the Mean Absolute Error (MAE).

[0115] (3) latent loss:

[0116] The latent loss is used to measure the difference between the latent vector of the real image and the latent vector of the reconstructed image. Its goal is to ensure that the latent vector of the reconstructed image is similar to the latent vector of the real image, thereby improving the performance of the model. The last convolutional layer of the discriminator is used to extract the latent representation of x and . The expression of the latent loss is shown in Equation 12.

[0117]

[0118] wherein, represents the L2 loss, that is, the Mean Squared Error (MSE).

[0119] The total loss of the last sea ice detection module is the weighted sum of the above three losses, as shown in Equation 13. adv con lat is a weighting parameter used to adjust the dominant weight of each loss part in the overall objective function:

[0120] L = w adv L adv + w con L con + w lat L lat (13)

[0121] The sea ice concentration retrieval module uses the MSE function, as shown in Equation 14:

[0122]

[0123] ​​The loss function is used to measure the difference between the model sea ice prediction value and the real sea ice value, y ~ p y representing sampling from the real data distribution.

[0124] According to the above-mentioned DDM data screening method based on the anomaly detection network, a graphics workstation with a CPU of Intel E-2124 main frequency of 3.3 GHz and a single NVIDIA 2080TI graphics card is used, the operating system is Ubuntu 16.04, and the deep learning framework of Pytorch 2.0 is used for verification.

[0125] The experimental parameters are as follows: the experimental learning rate is set to 0.0001, 500 epochs of training are performed on the DDM image data, and the input batch size is set to 64.

[0126] In order to verify the effectiveness and advancement of the present application, the present application is compared with four existing deep learning-based sea ice detection algorithms and sea ice concentration inversion algorithms, and the experiment proves that the sea ice detection accuracy of the algorithm of the present application reaches 0.9879, and the correlation coefficient of the sea ice concentration inversion reaches 0.9525, and the performance is better than that of the existing comparative algorithms. This shows that the present application realizes good sea ice detection and sea ice concentration inversion effect.

[0127] It should be noted that in the present application, relationship terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations.

[0128] The above is the preferred embodiment of the present application, and it should be pointed out that for ordinary skilled persons in the art, a number of improvements and refinements can be made without departing from the principles of the present application, and these improvements and refinements should also be considered within the scope of protection of the present application.

Claims

1. An anomaly detection driven sea ice semi-supervised multi-task inversion method, characterized by: The following steps are involved: S1. Obtain the DDM data from TDS-1 and input the obtained DDM data into the data preprocessing module. Through data screening, background noise subtraction and normalization, standard DDM data that can be used for sea ice multi-task inversion is obtained. S2. Design a multi-task grid structure, comprising a sea ice detection module and a sea ice concentration inversion module; use the seawater DDM data from the standard DDM data obtained in S1 as input to the sea ice detection module in the multi-task grid structure; perform deep feature extraction on the input seawater DDM data, accurately distinguish between sea ice data and seawater data, and complete the sea ice detection task; The sea ice detection module is used for anomaly detection and includes a generator G(x) and a discriminator D(x); The DDM images of seawater reflection are input into the sea ice detection module as normal samples for training, and the sea ice DDM images are used as abnormal samples; During the training process, the generator G(x) and the discriminator D(x) of the sea ice detection module compete with each other, wherein the generator G(x) is used to generate a reconstructed DDM image that is closer to the real DDM image x. So that the discriminator D(x) cannot distinguish true from false, the discriminator D(x) is used to distinguish x from The sea ice concentration inversion module includes a discriminant matching unit and an LSTM network; The discrimination and matching unit matches the output of the sea ice detection module with the original input image. The data that is sea ice data and is judged as abnormal by the sea ice detection module is used as the input of the LSTM network. The LSTM network receives the output of the matching and discrimination unit and obtains the high-dimensional feature representation of the corresponding DDM data through the inter-module data feature sharing mechanism to complete the sea ice concentration inversion task. The LSTM network consists of two stacked LSTM layers. Each LSTM layer is considered to perform sequence modeling on the input data. Stacking multiple layers allows the model to capture more complex sequence dependencies. Each LSTM layer includes 128 hidden units, each of which contains a memory unit. The LSTM includes three key gating mechanisms: a forget gate, an input gate, and an output gate to control the flow and update of cell states. S3. The high-dimensional features of the seawater DDM data extracted by the sea ice detection module and the module output results are input into the sea ice concentration inversion module through a potential deep learning guidance mechanism. The learned features are used to infer the sea ice concentration and complete the sea ice concentration inversion task.

2. The anomaly detection driven sea ice semi-supervised multi-task inversion method according to claim 1 is characterized in that: The screening conditions for the data screening work described in S1 are: ① Data with latitudes greater than 55 degrees north and south; ② Data with satellite incidence angle less than 35 degrees; ③Peak signal-to-noise ratio is greater than 0dB; ④Antenna gain is greater than 3dB; ⑤The sea ice concentration is 0.15 as the dividing line.

3. The anomaly detection driven sea ice semi-supervised multi-task inversion method according to claim 2 is characterized in that: The background noise subtraction described in S1 specifically includes the following: The calculation formula for the noise floor is: Based on formula (1), the formula for subtracting the background noise of each DDM data is: DDM remove_noise =DDM(τ,f)-Noise (2) Where Noise represents the noise value; τ1 and τ2 represent the pixel limits within the delay frequency band; f1 and f2 represent the pixel limits within the Doppler frequency band; X represents the total number of pixels within the noise box; and DDM(τ, f) represents the value at a specific position in the original DDM data in the delay and Doppler frequency directions.

4. The anomaly detection driven sea ice semi-supervised multi-task inversion method according to claim 3 is characterized in that: The normalization operation described in S1 specifically refers to normalizing each DDM according to its peak power. The normalization calculation formula is: Among them, DDM(τ,f) represents the value of a specific position of the original DDM data in the direction of time delay and Doppler frequency, DDM maxnosie The DDM value represents the peak power after subtracting the noise floor.

5. The anomaly detection driven sea ice semi-supervised multi-task inversion method according to claim 1 is characterized in that: The generator G(x) consists of an encoder and a decoder; the encoder maps the input image to a latent vector in the latent space by downsampling and extracts an abstract feature representation; each layer of the encoder contains a convolutional layer, a BatchNorm layer, and a LeakyReLU activation function, which reads its input data into the network and outputs a latent vector in the latent space; The decoder maps the latent vector of the latent space back to the original image space by upsampling, that is, reconstructing the abstract feature representation into an image again, and finally generating a realistic image; The decoder adopts the structure of transposed convolution and BatchNorm layer, which corresponds one-to-one to the structure of the encoder; transposed convolution is used to upsample the input latent space vector to increase the size of the feature map; the decoder uses Tanh activation function in the output layer, and ReLU activation function in other layers.

6. The anomaly detection driven sea ice semi-supervised multi-task inversion method according to claim 5 is characterized in that: The D(x) is composed of a convolutional layer, a BatchNorm layer, and a LeakyReLU activation function. The discriminator D(x) is used to judge the authenticity of the reconstructed image generated by the generator G(x) and the input real image, and classify them. The goal of the discriminator D(x) is to get a high score on the real image and a low score on the reconstructed image. The discriminator D(x) also acts as a feature extractor, taking the intermediate layer output of the discriminator as features, and then calculating the feature matching loss between the generated image and the real image.

7. The anomaly detection driven sea ice semi-supervised multi-task inversion method according to claim 1 is characterized in that: The potential deep learning guidance mechanism described in S3 is implemented through skip connections, which enable information to jump and pass in the network, connecting the feature map of the last layer in the encoder with the feature map of the corresponding layer in the decoder to pass more detailed information and spatial context, and recover richer detail information in the decoder stage.

8. The anomaly detection driven sea ice semi-supervised multi-task inversion method according to claim 1 is characterized in that: The forget gate is used to determine how much of the unit state at the previous moment is retained to the unit state at the current moment. The specific function is expressed as: f t =σ(W f ·[h t-1 ,x t ]+b f ) (4) Among them, f t represents the output value of the forget gate; σ represents the sigmoid layer; w f represents the weight matrix of the forget gate; x t Represents the input of the current time step; h t-1 represents the hidden state of the previous time step; [h t-1 ,x t ] represents the vector formed by connecting the hidden state of the previous time step and the input of the current time step; b f represents the bias vector; The input gate uses learned weights to determine which information should enter the cell state. It consists of two parts: a sigmoid layer and a tanh layer. The sigmoid layer determines what value to update, and the tanh layer updates the information that needs to be updated to the cell state. The specific function is expressed as follows: i t =σ(W i ·[h t -1,x t ])+b i (5) Among them, i t Represents the output value of the input gate; represents the candidate value of the input gate; W i 、W f represents the weight matrix; b i 、b c represents the bias vector; New cell state C t The memory in is composed of the old memory left after the memory cells of the previous moment forget the useless memory and the useful information filtered out by the input gate as the new memory. The function of the new cell state is expressed as follows: The output gate is used to determine which information should be output to the next layer or as the final output. The specific function is as follows: the t =σ(W o [h t-1 ,x t ]+b o ) (8) h t =o t *fishy(C) t ) (9) Among them, t Represents the output value of the output gate; h t represents the hidden state of the current time step; The hidden state is part of the LSTM network output and is used to carry the information of the network at the current time step. The hidden state is useful information extracted from the cell state and is used for network prediction or input of the next time step.

Images