Chlorophyll concentration completion method and system based on abnormal attention and FNO

Abstract

The present invention belongs to the technical field of image completion, and discloses a chlorophyll concentration completion method and system based on abnormal attention and FNO. The system includes a spatial domain dual-stream attention network and a frequency domain fusion completion network. The spatial domain dual-stream attention network takes the weekly average chlorophyll concentration field and the damaged chlorophyll concentration field as input, and outputs the stable feature vector and the abnormal feature vector in the spatial domain through the spatial stable attention module and the spatial abnormal attention module at the same time; the frequency domain fusion completion network takes the stable feature vector and the abnormal feature vector in the spatial domain as input, and outputs the completed chlorophyll concentration field after fusion completion in the frequency domain, and is accompanied by the reconstructed weekly average chlorophyll concentration field and the damaged chlorophyll concentration field for discriminant training. The present invention improves the accuracy of completion.

Description

Technical Field

[0001] The present invention belongs to the technical field of image completion, and in particular relates to a chlorophyll concentration completion method and system based on abnormal attention and FNO. Background Art

[0002] The completion of the spatiotemporal data field of chlorophyll concentration on the sea surface has been an important direction in marine science in recent years. Traditional methods are usually based on generative models, such as generative adversarial networks, reconstruction models, and neural network models such as encoder-decoders that use historical data to directly estimate and complete the chlorophyll concentration field. Traditional methods have the following problems: unstable and inaccurate completion. The reasons are: on the one hand, traditional methods usually use a one-time completion method to directly complete the missing area. On the other hand, the numerical range of the chlorophyll concentration field is usually large, and directly estimating such a large numerical range will make the completion process very unstable, and it is difficult to accurately estimate the value of each point. Compared with traditional methods, the chlorophyll concentration field completion method based on deep neural networks uses a large amount of historical ocean data to estimate the distribution of spatiotemporal data of chlorophyll concentration.

[0003] At present, the cutting-edge method for ocean chlorophyll concentration completion based on deep learning is the ocean image completion method based on the anomaly repair network, which adopts a "coarse to fine" image completion mechanism. Most of them adopt a two-stage completion method. In the first stage, the monthly average of ocean chlorophyll concentration is used to estimate the weekly average. In the second stage, the deviation of the missing part is estimated with the help of the deviation between the non-missing part of the daily data of ocean chlorophyll concentration to be completed and the weekly average, and then it is superimposed with the weekly average to obtain the completed daily chlorophyll concentration field.

[0004] However, the above methods have the following problems: First, only the representation learning of significant information is carried out, and the mining of non-significant (abnormal) information is insufficient, which limits the accuracy of completion. For example, most methods use traditional significant representation strategies such as convolution and attention mechanism to complete chlorophyll concentration, ignoring that these representation strategies will smooth non-significant information (such as local anomalies that do not meet the mean value within the period), thereby limiting the accuracy of the final completion. Second, relying on the spatial domain representation strategy in semantic and visual tasks, it is difficult to fully represent marine data without clear semantic information, resulting in poor representation quality. For example, the representation learning strategies used by most methods are mostly inspired by the traditional image completion field, and all rely on clear semantic and visual information in the image to carry out representation learning. However, chlorophyll concentration images belong to marine scientific data. Their composition not only lacks clear semantic information, but also behaves as a complex chaotic system. Such spatial domain representation strategies are difficult to fully represent them, limiting the quality of the representation results. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a chlorophyll concentration completion method and system based on abnormal attention and FNO. (1) A spatial domain dual-stream attention network is designed for the chlorophyll concentration completion task, which solves the problem that the existing method only carries out representation learning of significant information, but does not fully mine non-significant (abnormal) information, thus limiting the accuracy of completion. (2) In the chlorophyll concentration completion task, the Fourier neural operator is introduced and combined with the designed spatial domain dual-stream attention network, which solves the problem that the original method relies on the spatial domain representation strategy in semantic and visual tasks, and is difficult to fully represent ocean data without clear semantic information, resulting in poor representation quality.

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0007] First, the present invention provides a chlorophyll concentration complement method based on abnormal attention and FNO, comprising the following steps:

[0008] Step S1, input data:

[0009] Input data includes the broken chlorophyll concentration field SST cor , Weekly mean chlorophyll concentration field SST ave :

[0010] Step S2: Dual-stream attention processing in spatial domain:

[0011] First, SST is extracted through different convolutions cor The spatial representation vector Emb sta 、Emb ano , extract SST ave The spatial representation vector Emb ave , and then strengthen Emb by stabilizing attention sta Middle and Emb ave Latent data patterns with similar correlations lead to stable eigenvectors in the spatial domain ; At the same time, strengthen Emb through abnormal anti-attention ano Emb ave Latent data patterns with negative correlations, resulting in abnormal feature vectors in the spatial domain ;

[0012] Step S3: frequency domain fusion completion:

[0013] for , , using the Fourier neural operator FNO in the frequency domain, after fusion and completion, the completed chlorophyll concentration field SST is output rec , and the reconstructed weekly mean chlorophyll concentration field is obtained , damaged chlorophyll concentration field ;SST rec and the real chlorophyll concentration field SST gro Use discriminator D2 for discriminative training, and SST ave Use discriminator D1 for discriminative training, and SST cor Use discriminator D3 for discriminative training.

[0014] Furthermore, in step S2, the stable eigenvector in the spatial domain The calculation of is as follows:

[0015] (1);

[0016] (2);

[0017] (3);

[0018] in, represents the 1st, 2nd, ..., ...i, ..., hth heads in the multi-head attention mechanism, and h represents the total number of heads. represents the weight matrix used for the concatenation operation, , and Represents query, key and value respectively, , and is the corresponding weight matrix, yes The dimension of the vector in ; where and From Emb sta ,and From Emb ave , the softmax function is used to calculate and Similar correlations between

[0019] Abnormal eigenvectors in the spatial domain The calculation of is as follows:

[0020] (4);

[0021] (5);

[0022] in and From Emb ano ,and From Emb ave ;

[0023] Formula (5) indicates the use of ReverseAttention, which is as follows:

[0024] (6);

[0025] pass Perform similarity correlation inversion operation.

[0026] Furthermore, the Fourier neural operator FNO includes Fourier transform, filter layer, and inverse Fourier transform. Step S3 is specifically as follows:

[0027] Step S31, obtaining frequency domain information:

[0028] Input feature vector , , first apply Fourier transform to convert it to Fourier frequency domain, then use 2 corresponding filter layers , Filter and obtain stable frequency domain information Sign sta , abnormal frequency domain information Sign ano ;

[0029] Step S32: frequency domain information fusion:

[0030] Sign the stable frequency domain information sta , abnormal frequency domain information Sign ano At the same time, a layer of complex channel convolution is input to achieve the fusion of the two information in the frequency domain, and the fused frequency domain information Sign is output. fus ;

[0031] Step S33: chlorophyll concentration field fusion and completion:

[0032] The fused frequency domain information Sign fus After inverse Fourier transform, the frequency domain is restored back to the spatial domain to obtain the representation vector Emb rec , and then use 2D convolution to Emb rec Perform nonlinear enhancement and output the completed chlorophyll concentration field SST rec ;

[0033] Step S34: Reconstruct the weekly average chlorophyll concentration field and damaged chlorophyll concentration field :

[0034] Stable frequency domain information Sign sta , abnormal frequency domain information Sign ano After inverse Fourier transform and 2D convolution, the reconstructed weekly average chlorophyll concentration field is output. and damaged chlorophyll concentration field ;

[0035] Step S35, discriminator training:

[0036] SST rec and the real chlorophyll concentration field SST gro Use discriminator D2 for discriminative training, and SST ave Use discriminator D1 for discriminative training, and SST cor Use discriminator D3 for discriminative training.

[0037] Furthermore, the formula in step 3 is as follows:

[0038] (7);

[0039] (8);

[0040] in, , The filter layers are , The fully connected layer parameters that need to be trained and learned in Represents Fourier transform, and the process is as follows:

[0041] (9);

[0042] Among them, j is an imaginary unit, f(x,y) represents the pixel point in the input original image, x and y are the horizontal and vertical coordinate values, M and N are the length and width of the original image, and F(u,v) is the frequency domain signal.

[0043] Furthermore, during training, the loss function is as follows:

[0044] (17);

[0045] It is divided into two parts: ① represents the reconstruction loss of this model, and ② represents the discrimination loss of this model. They are introduced below:

[0046] ①Reconstruction loss loss recons The specific calculation is as follows:

[0047] (18);

[0048] in, is the L2 norm, and the mask is SST cor The binary mask of:

[0049] (19);

[0050] ②Discrimination loss discri The specific calculation is as follows:

[0051] (20);

[0052] (twenty one);

[0053] (twenty two);

[0054] (twenty three);

[0055] Among them, D1, D2, and D3 represent discriminators. , , are the parameters that need to be learned, SST ave is the weekly average chlorophyll concentration field, is the reconstructed weekly mean chlorophyll concentration field, SST rec To complete the chlorophyll concentration field, SST gro is the real chlorophyll concentration field, SST cor is the damaged chlorophyll concentration field, is the reconstructed damaged chlorophyll concentration field.

[0056] Secondly, the present invention provides a chlorophyll concentration completion system based on abnormal attention and FNO, which is used to implement the chlorophyll concentration completion method based on abnormal attention and FNO as described above, including a spatial domain dual-stream attention network and a frequency domain fusion completion network, wherein the spatial domain dual-stream attention network converts the weekly average chlorophyll concentration field SST ave , damaged chlorophyll concentration field SST cor As input, the spatial stability attention module and the spatial anomaly attention module simultaneously output stable feature vectors in the spatial domain. , abnormal feature vector ;

[0057] The frequency domain fusion completion network transforms the feature vector , As input, after fusion and completion in the frequency domain, the completed chlorophyll concentration field SST is output rec , and the reconstructed weekly mean chlorophyll concentration field is obtained , damaged chlorophyll concentration field ;SST rec and the real chlorophyll concentration field SST gro Use discriminator D2 for discriminative training, and SSTave Use discriminator D1 for discriminative training, and SST cor Use discriminator D3 for discriminative training.

[0058] Compared with the prior art, the present invention has the advantages of:

[0059] (1) A spatial domain dual-stream attention network is designed for the chlorophyll concentration completion task, which fully exploits the mining of non-significant (abnormal) information and improves the completion accuracy. The present invention uses stable attention to strengthen the potential data patterns in the damaged chlorophyll concentration field that have similar correlations with the weekly average chlorophyll concentration field in the spatial domain dual-stream attention network by inputting the weekly average chlorophyll concentration field and the damaged chlorophyll concentration field, thereby obtaining a stable feature vector in the spatial domain. Abnormal anti-attention is also used to strengthen the potential data patterns in the damaged chlorophyll concentration field that have negative correlations with the weekly average chlorophyll concentration field, thereby obtaining an abnormal feature vector in the spatial domain. This design fully guarantees the mining of both significant information and non-significant (abnormal) information, thereby improving the completion accuracy of the model.

[0060] (2) In the task of chlorophyll concentration completion, the Fourier neural operator is introduced and combined with the designed spatial domain dual-stream attention network, and the frequency domain is fully utilized to carry out the representation learning of ocean data, thereby improving the representation quality. The present invention extends the Fourier neural operator to the field of chlorophyll concentration completion, combines it with the proposed spatial domain dual-stream attention network, and adjusts the low-pass filter implemented by the roll operator in the Fourier neural operator using a neural network to address the problem in this field, and changes it to a fully connected layer to implement an adaptive filter, fully representing the stable feature vectors and abnormal feature vectors in the spatial domain from the frequency domain and obtaining stable frequency domain information and abnormal frequency domain information, and then simultaneously inputs the stable frequency domain information and abnormal frequency domain information into a layer of complex channel convolution to achieve the fusion of the two information in the frequency domain, and obtain the fused frequency domain information, and fully carry out effective and sufficient representation learning of ocean data in the frequency domain, and obtain effective frequency information to complete the completion task, thereby improving the representation quality of ocean data. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0062] Figure 1 is a system structure diagram of the present invention;

[0063] Figure 2 Schematic diagram of the Fourier neural operator of the present invention; Figure 2 FT stands for Fourier transform, R stands for Fourier convolution operator, FT -1 represents the inverse Fourier transform, W represents the local linear layer, σ is the activation function, and + represents element-wise addition. DETAILED DESCRIPTION

[0064] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0065] Combination Figure 1 As shown in the figure, the present invention mainly designs two modules, the spatial domain dual-stream attention network and the frequency domain fusion completion network. ① The spatial domain dual-stream attention network converts the weekly average chlorophyll concentration field SST ave , damaged chlorophyll concentration field SST cor As input, the spatial stability attention module and the spatial anomaly attention module simultaneously output stable feature vectors in the spatial domain. , abnormal feature vector ② The frequency domain fusion completion network transforms the feature vector , As input, after fusion and completion in the frequency domain, the completed chlorophyll concentration field SST is output rec , and the reconstructed weekly mean chlorophyll concentration field is obtained , damaged chlorophyll concentration field Finally, SST rec and the real chlorophyll concentration field SST gro Use discriminator D2 for discriminative training, and SST ave Use discriminator D1 for discriminative training, and SST cor Use discriminator D3 for discriminative training.

[0066] The following is a detailed description of the chlorophyll concentration supplementation method based on abnormal attention and FNO of the present invention in conjunction with the accompanying drawings, comprising the following steps:

[0067] Step S1, input data:

[0068] Input data includes the broken chlorophyll concentration field SST cor , Weekly mean chlorophyll concentration field SST ave ,The input leaf green concentration field data are all images, and the purpose is to complete the missing images.

[0069] Step S2: Dual-stream attention processing in spatial domain:

[0070] First, different 2D convolutions are applied to pre-extract SST corThe spatial representation vector Emb sta 、Emb ano , extract SST ave The spatial representation vector Emb ave .Em sta and Emb ano Represents different learned representations, Emb sta After sufficient training, what is learned is a "stable" representation vector, Emb ano What is learned after sufficient training is the "abnormal" representation vector.

[0071] Note: Although the input of both is the same SST cor , but the 2D convolutions are not the same, they are independent of each other, so after sufficient training, the representation vectors obtained have different meanings.

[0072] Then, strengthen Emb by stabilizing attention sta Emb ave Latent data patterns with similar correlations lead to stable eigenvectors in the spatial domain ; At the same time, strengthen Emb through abnormal anti-attention ano Emb ave Latent data patterns with negative correlations, resulting in abnormal feature vectors in the spatial domain .

[0073] Among them, the stable eigenvector in the spatial domain The calculation of is as follows:

[0074] (1);

[0075] (2);

[0076] (3);

[0077] in, represents the 1st, 2nd, ..., ...i, ..., hth heads in the multi-head attention mechanism, and h represents the total number of heads. represents the weight matrix used for the concatenation operation, , and Represents query, key and value respectively, , and is the corresponding weight matrix, yes The dimension of the vector in ; where and From Emb sta ,and From Emb ave , the softmax function is used to calculate and Similar correlations between .

[0078] Abnormal eigenvectors in the spatial domain The calculation of is as follows:

[0079] (4);

[0080] (5);

[0081] in and From Emb ano ,and From Emb ave .

[0082] Note: Formula (5) indicates the use of ReverseAttention, which is as follows:

[0083] (6);

[0084] pass Perform similar correlation inversion operations and pay more attention to and Potential data patterns with lower similarity.

[0085] therefore, Enhanced Emb sta Emb ave The design of the spatial anomaly attention module solves the problem that the original method only carries out representation learning of significant information, but does not fully mine non-significant (abnormal) information, thus limiting the accuracy of completion.

[0086] Step S3: frequency domain fusion completion:

[0087] for , , using the Fourier neural operator FNO in the frequency domain, after fusion and completion, the completed chlorophyll concentration field SST is output rec , and the reconstructed weekly mean chlorophyll concentration field is obtained , damaged chlorophyll concentration field ;SST rec and the real chlorophyll concentration field SST gro Use discriminator D2 for discriminative training, and SST aveUse discriminator D1 for discriminative training, and SST cor Use discriminator D3 for discriminative training.

[0088] It should be noted that the Fourier neural operator FNO includes three parts: Fourier transform, filter layer, and inverse Fourier transform. Step S3 is as follows:

[0089] Step S31, obtaining frequency domain information:

[0090] Input feature vector , , first apply Fourier transform to convert it to Fourier frequency domain, then use 2 corresponding filter layers , Filter and obtain stable frequency domain information Sign sta , abnormal frequency domain information Sign ano .

[0091] Note: Filter layer , It is implemented by a fully connected layer neural network, which filters and retains the information of key frequency bands through adaptive learning of the neural network. The formula for the above steps is as follows:

[0092] (7);

[0093] (8);

[0094] in, , The filter layers are , The fully connected layer parameters that need to be trained and learned in Represents Fourier transform, and the process is as follows:

[0095] (9);

[0096] Where j is an imaginary unit, f(x,y) represents the pixel point in the input original image, x and y are coordinate variables in the spatial domain, which are the horizontal and vertical coordinate values ​​respectively, M and N are the length and width of the original image respectively, F(u,v) is the frequency domain signal, and u and v are two frequency variables in the frequency domain.

[0097] Step S32: frequency domain information fusion:

[0098] Sign the stable frequency domain information sta , abnormal frequency domain information Sign ano At the same time, a layer of complex channel convolution is input to achieve the fusion of the two information in the frequency domain, and the fused frequency domain information Sign is output.fus ; The process is as follows:

[0099] (10);

[0100] in, Represents a complex channel convolution operation.

[0101] Step S33: chlorophyll concentration field fusion and completion:

[0102] The fused frequency domain information Sign fus After inverse Fourier transform, the frequency domain is restored back to the spatial domain (to complete the subsequent completion work) to obtain the representation vector Emb rec , the process is as follows:

[0103] (11);

[0104] in, is the inverse Fourier transform, and the process is as follows:

[0105] (12);

[0106] Among them, j is the imaginary unit, f(x,y) is the original image, M and N are the length and width of the original image respectively, and F(u,v) is the frequency domain signal.

[0107] Then, use 2D convolution to embed rec Perform nonlinear enhancement and output the completed chlorophyll concentration field SST rec .

[0108] Step S34: Reconstruct the weekly average chlorophyll concentration field and damaged chlorophyll concentration field .

[0109] Similar to the application of inverse Fourier transform to transform Sign fus Restoring from the frequency domain to the spatial domain and subsequent operations, for stable frequency domain information Sign sta , abnormal frequency domain information Sign ano The same operation is performed, that is, after inverse Fourier transform and 2D convolution, the reconstructed weekly average chlorophyll concentration field is output and damaged chlorophyll concentration field .

[0110] It should be noted that the Fourier transform neural operator was first introduced by researchers to solve partial differential equations. The structure of the Fourier transform neural operator is shown in the figure below. Figure 2 As shown. FT stands for Fourier transform, R stands for Fourier convolution operator, represents the inverse Fourier transform, and W represents the local linear layer. Specifically, the traditional neural operator satisfies arrive The iterative relationship is solved, and the specific formula is as follows:

[0111] (13);

[0112] in, is the result obtained by Fourier transform neural operator. is the result of the previous iteration, is the activation function, is the kernel function learned from the data, is the parameter learned by the neural network and is defined as follows:

[0113] (14);

[0114] Among them, in order to combine the advantages of the convolution layer in the neural network being good at learning local information and the advantages of the Fourier transform being good at learning global information, the convolution operator R under the Fourier transform is proposed. Specifically, first, the kernel function When performing approximate learning, the Dependencies and limitations Formula (15) is a convolution operator, namely:

[0115] (15);

[0116] Then, applying the Fourier transform yields:

[0117] (16);

[0118] Among them, the Fourier convolution operator is a periodic function parameterized by a neural network.

[0119] Here, by extending the Fourier neural operator to the field of chlorophyll concentration completion and combining it with the spatial domain two-stream attention network, the proposed method for the chlorophyll concentration spatiotemporal data completion task can adaptively carry out effective and sufficient representation learning of ocean data in the frequency domain, solving the problem that the original method relies on the spatial domain representation strategy in semantic and visual tasks, which is difficult to fully represent ocean data without clear semantic information, resulting in poor representation quality. Note that the original method of the Fourier neural operator The neural network used in the paper is a low-pass filter, but in this method, there is no clear restriction on the high or low frequency of the significant and non-significant information involved. This method uses a fully connected layer without any low-pass or high-pass restriction to achieve the adaptive learning of the neural network and stable frequency domain information Signsta , abnormal frequency domain information Sign ano This is also a key design of the present invention to introduce Fourier neural operator into the field of chlorophyll concentration completion to address the field problem.

[0120] Step S35, discriminator training:

[0121] SST rec and the real chlorophyll concentration field SST gro Use discriminator D2 for discriminative training, and SST ave Use discriminator D1 for discriminative training, and SST cor Use discriminator D3 for discriminative training.

[0122] Among them, a group of discriminators (D1, D2, D3) designed using a multi-task mechanism fully ensure that the model learns: stable frequency domain information Sign sta , abnormal frequency domain information Sign ano quality and robustness.

[0123] The design of the spatial anomaly attention module solves the problem that the original method only conducts representation learning of significant information, and does not fully mine non-significant (abnormal) information, which limits the accuracy of completion. Secondly, by extending the Fourier neural operator to the field of chlorophyll concentration completion and combining it with the spatial domain dual-stream attention network, the method proposed for the chlorophyll concentration spatiotemporal data completion task can adaptively conduct effective and sufficient representation learning of ocean data in the frequency domain, solving the problem that the original method relies on the spatial domain representation strategy in semantic and visual tasks, and is difficult to fully represent ocean data without clear semantic information, resulting in poor representation quality.

[0124] During training, the loss function of the present invention is as follows:

[0125] (17);

[0126] It is divided into two parts: ① represents the reconstruction loss of this model, and ② represents the discrimination loss of this model, which are introduced below.

[0127] ①Reconstruction loss loss recons The specific calculation is as follows:

[0128] (18);

[0129] in, is the L2 norm, and the mask is SST cor The binary mask of:

[0130] (19);

[0131] ②Discrimination loss discri The specific calculation is as follows:

[0132] (20);

[0133] (twenty one);

[0134] (twenty two);

[0135] (twenty three);

[0136] Among them, D1, D2, and D3 represent discriminators. , , are the parameters that need to be learned, SST ave is the weekly average chlorophyll concentration field, is the reconstructed weekly mean chlorophyll concentration field, SST rec To complete the chlorophyll concentration field, SST gro is the real chlorophyll concentration field, SST cor is the damaged chlorophyll concentration field, is the reconstructed damaged chlorophyll concentration field.

[0137] As another embodiment of the present invention, a chlorophyll concentration completion system based on abnormal attention and FNO is provided, which can be used to implement the chlorophyll concentration completion method based on abnormal attention and FNO as described above, such as Figure 1 As shown, it includes a spatial domain dual-stream attention network and a frequency domain fusion completion network. The spatial domain dual-stream attention network converts the weekly average chlorophyll concentration field SST ave , damaged chlorophyll concentration field SST cor As input, the spatial stability attention module and the spatial anomaly attention module simultaneously output stable feature vectors in the spatial domain. , abnormal feature vector .

[0138] The frequency domain fusion completion network transforms the feature vector , As input, after fusion and completion in the frequency domain, the completed chlorophyll concentration field SST is output rec , and the reconstructed weekly mean chlorophyll concentration field is obtained , damaged chlorophyll concentration field ;SST recand the real chlorophyll concentration field SST gro Use discriminator D2 for discriminative training, and SST ave Use discriminator D1 for discriminative training, and SST cor Use discriminator D3 for discriminative training.

[0139] The functional implementation and data processing process of each module can be found in the previous section on the chlorophyll concentration completion method based on abnormal attention and FNO, which will not be repeated here.

[0140] In summary, the present invention (1) designs a spatial domain dual-stream attention network for the chlorophyll concentration completion task, fully exploits non-significant (abnormal) information, and improves the completion accuracy. For example, the original method uses traditional significant representation strategies such as convolution to complete chlorophyll concentration, ignoring that these representation strategies will smooth non-significant information (such as local anomalies that do not meet the intra-period mean conditions), such as average pooling and maximum pooling operations used in convolution operations. Secondly, the above method directly superimposes local anomalies and weekly averages, and the huge numerical difference between the two further aggravates the smoothing of abnormal information, limiting the accuracy of the final completion.

[0141] This method uses the weekly mean chlorophyll concentration field SST as input ave , damaged chlorophyll concentration field SST cor In the spatial domain dual-stream attention network, stable attention is used to strengthen the damaged chlorophyll concentration field SST cor The average chlorophyll concentration field SST ave Latent data patterns with similar correlations lead to stable eigenvectors in the spatial domain . The abnormal anti-attention was used to strengthen the damaged chlorophyll concentration field SST cor The average chlorophyll concentration field SST ave Latent data patterns with negative correlations, resulting in abnormal feature vectors in the spatial domain This design fully guarantees the mining of both significant information and non-significant (abnormal) information, thus improving the completion accuracy of the model.

[0142] (2) In the chlorophyll concentration completion task, the Fourier neural operator is introduced and combined with the designed spatial domain two-stream attention network, which makes full use of the frequency domain to carry out representation learning of ocean data and improves the representation quality. For example, the representation learning strategy used in the original method was inspired by the traditional image completion field and relied on clear semantic and visual information in the image to carry out representation learning. However, marine science data not only lacks clear semantic information, but also appears as a complex chaotic system. Such spatial domain representation strategies are difficult to fully represent and learn it, resulting in poor representation quality. This method extends the Fourier neural operator to the field of chlorophyll concentration completion, combines it with the proposed spatial domain two-stream attention network, and performs a special analysis of the Fourier neural operator in this field. The low-pass filter implemented using a neural network was adjusted to a fully connected layer to implement an adaptive filter (i.e. , ), fully characterizes the stable feature vector in the learning space domain from the frequency domain , abnormal feature vector And get stable frequency domain information Sign sta , abnormal frequency domain information Sign ano , then Sign sta 、Sign ano At the same time, a layer of complex channel convolution is input to achieve the fusion of the two information in the frequency domain, and the fused frequency domain information Sign is obtained. fus , effectively and fully carry out characterization learning of ocean data in the frequency domain, obtain effective frequency information to complete the completion task, and improve the characterization quality of ocean data.

[0143] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Any changes, modifications, additions or substitutions made by ordinary technicians in this technical field within the essential scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A chlorophyll concentration complementation method based on abnormal attention and FNO, characterized in that: The following steps are involved: Step S1, input data: Input data includes the broken chlorophyll concentration field SST cor , Weekly mean chlorophyll concentration field SST ave : Step S2: Dual-stream attention processing in spatial domain: First, SST is extracted through different convolutions cor The spatial representation vector Emb sta 、Emb ano , extract SST ave The spatial representation vector Emb ave , and then strengthen Emb by stabilizing attention sta Middle and Emb ave Latent data patterns with similar correlations lead to stable eigenvectors in the spatial domain At the same time, Emb is strengthened through abnormal anti-attention ano Emb ave Latent data patterns with negative correlations, resulting in abnormal feature vectors in the spatial domain In step S2, the stable eigenvector in the spatial domain The calculation of is as follows: Among them, head1, head2, ...head i , ..., head h represents the 1st, 2nd, ..., ...i, ..., hth heads in the multi-head attention mechanism, h represents the total number of heads, and W sta represents the weight matrix used for the concatenation operation, Q (1) , K (1) and V (1) Represents query, key and value respectively, and is the corresponding weight matrix, d k It's K (1) The dimension of the vector in (1) and V (1) From Emb sta , and K (1) From Emb ave , the softmax function is used to calculate Q (1) and K (1) Similar correlations between Abnormal eigenvectors in the spatial domain The calculation of is as follows: Where Q (2) and V (2) From Emb ano , and K (2) From Emb ave ; Formula (5) indicates the use of ReverseAttention, which is as follows: pass Perform similarity correlation inversion operation; Step S3: frequency domain fusion completion: for The Fourier neural operator FNO is used in the frequency domain, and the completed chlorophyll concentration field SST is output after fusion and completion. rec , and the reconstructed weekly mean chlorophyll concentration field is obtained Broken chlorophyll concentration field SST rec and the real chlorophyll concentration field SST gro Use discriminator D2 for discriminative training, and SST ave Use discriminator D1 for discriminative training, and SST cor Use discriminator D3 for discriminative training; Among them, the Fourier neural operator FNO includes Fourier transform, filter layer, and inverse Fourier transform. Step S3 is as follows: Step S31, obtaining frequency domain information: Input feature vector First, a Fourier transform is applied to convert it to the Fourier frequency domain, and then two corresponding filter layers R are used. sta (·;θ1),R ano (·; θ2) to filter and obtain stable frequency domain information Sign sta , Abnormal frequency domain information Sign ano ; Step S32: frequency domain information fusion: Sign the stable frequency domain information sta , Abnormal frequency domain information Sign ano At the same time, a layer of complex channel convolution is input to achieve the fusion of the two information in the frequency domain, and the fused frequency domain information Sign is output. fus ; Step S33, chlorophyll concentration field fusion completion: The fused frequency domain information Sign fus After inverse Fourier transform, the frequency domain is restored back to the spatial domain to obtain the representation vector Emb rec , and then use 2D convolution to Emb rec Perform nonlinear enhancement and output the completed chlorophyll concentration field SST rec ; Step S34: Reconstruct the weekly average chlorophyll concentration field and the damaged chlorophyll concentration field Stable frequency domain information Sign sta , Abnormal frequency domain information Sign ano After inverse Fourier transform and 2D convolution, the reconstructed weekly average chlorophyll concentration field is output. and the damaged chlorophyll concentration field Step S35, discriminator training: SST rec and the real chlorophyll concentration field SST gro Use discriminator D2 for discriminative training, and SST ave Use discriminator D1 for discriminative training, and SST cor Use discriminator D3 for discriminative training.

2. The chlorophyll concentration complementation method based on abnormal attention and FNO according to claim 1, characterized in that: The formula in step 3 is as follows: Among them, θ1 and θ2 are the filter layer R sta (·;θ1),R ano (·; θ2) is the fully connected layer parameter that needs to be trained and learned. FT(·) represents Fourier transform. The process is as follows: Among them, j is an imaginary unit, f(x,y) represents the pixel point in the input original image, x and y are the horizontal and vertical coordinate values, M and N are the length and width of the original image, and F(u,v) is the frequency domain signal.

3. The chlorophyll concentration complementation method based on abnormal attention and FNO according to claim 1, characterized in that: During training, the loss function is as follows: L 总 It is divided into two parts: ① represents the reconstruction loss of this model, and ② represents the discrimination loss of this model. They are introduced below: ①Reconstruction loss loss recons The specific calculation is as follows: Among them, ||·||2 is the L2 norm, and mask is SST cor The binary mask of: ②Discrimination loss discri The specific calculation is as follows: Among them, D1, D2, and D3 represent the discriminator, θ1, θ2, and θ3 are the parameters they need to learn, and SST ave is the weekly mean chlorophyll concentration field, is the reconstructed weekly mean chlorophyll concentration field, SST rec To complete the chlorophyll concentration field, SST gro is the real chlorophyll concentration field, SST cor is the damaged chlorophyll concentration field, is the reconstructed damaged chlorophyll concentration field.

4. The chlorophyll concentration complement system based on abnormal attention and FNO is characterized by: The method for completing chlorophyll concentration based on abnormal attention and FNO as described in any one of claims 1 to 3 comprises a spatial domain dual-stream attention network and a frequency domain fusion completion network, wherein the spatial domain dual-stream attention network converts the weekly average chlorophyll concentration field SST ave , damaged chlorophyll concentration field SST cor As input, the spatial stability attention module and the spatial anomaly attention module simultaneously output stable feature vectors in the spatial domain. Abnormal feature vector The frequency domain fusion completion network transforms the feature vector As input, after fusion and completion in the frequency domain, the completed chlorophyll concentration field SST is output rec , and the reconstructed weekly mean chlorophyll concentration field is obtained Broken chlorophyll concentration field SST rec and the real chlorophyll concentration field SST gro Use discriminator D2 for discriminative training, and SST ave Use discriminator D1 for discriminative training, and SST cor Use discriminator D3 for discriminative training.

Images