Sea surface chlorophyll-a concentration early warning method and device based on multi-modal adversarial fusion

Abstract

This invention discloses a method and device for early warning of sea surface chlorophyll a concentration based on multimodal adversarial fusion. The method includes: inputting the basic representations of discretized sea surface physical field data and sea surface wind field data into an adversarial network to achieve multimodal information fusion; using the fused feature operators as objective influencing factors affecting changes in sea surface chlorophyll a concentration; fusing the objective and subjective influencing factors to obtain a feature representation of changes in sea surface chlorophyll a concentration; and using the feature representation to obtain the sea surface chlorophyll a concentration value at future times, thereby achieving prediction and early warning of sea surface chlorophyll a concentration. The device includes a processor and a memory. This invention improves the accuracy of sea surface chlorophyll a concentration prediction and can provide early warning information on sea surface chlorophyll a concentration to coastal areas 24 hours in advance, thereby preventing the occurrence of harmful red tides and algal blooms.

Description

Technical Field

[0001] This invention relates to the field of sea surface chlorophyll a concentration early warning, and in particular to a method and device for sea surface chlorophyll a concentration early warning based on multimodal adversarial fusion. Background Technology

[0002] With the continuous changes and interactions of physical quantities such as sea surface wind speed, air pressure, and geopotential, cyclones of varying sizes form on average 3-5 degrees of latitude away from the equator in tropical regions. These cyclones are typically accompanied by strong winds, torrential rains, and storm surges. Under the influence of upwelling and vertical mixing of cyclones, they bring terrestrial materials, such as colored dissolved organic matter and suspended sediments, to the ocean surface, hindering phytoplankton photosynthesis and reducing the concentration of chlorophyll a in the ocean. Changes in sea surface chlorophyll a concentration are often closely related to past concentrations and the time-frequency characteristics of the cyclone and its physical field changes at the current moment. Therefore, accurately predicting the future concentration of sea surface chlorophyll a is crucial for improving the ability of East Asia and coastal areas to prevent and mitigate marine algal blooms and red tide disasters. The data required for predicting sea surface chlorophyll a concentration mainly includes: latitude of the sea surface cyclone center, longitude of the sea surface cyclone center, average sea surface air pressure, average sea surface wind speed, and changes in the wind speed component of the surrounding sea area. These data contain a lot of hidden information, which has attracted the attention of many researchers to the information fusion mining of multiphysics.

[0003] Traditional methods for predicting sea surface chlorophyll a concentration utilize physical principles such as kinetics and thermodynamics to construct data models, and then use these models to predict sea surface chlorophyll a concentration. In recent years, with the continuous optimization of deep learning and artificial intelligence, more and more deep learning networks have been applied to various fields, greatly facilitating people's lives. More researchers are beginning to use deep learning to predict sea surface chlorophyll a concentration, for example, using backpropagation (BP) neural networks. [1-2] Convolutional Neural Networks, Long Short-Term Memory Networks [3-4] However, these methods only consider temporal context information, ignoring the structured relationships between data and thus losing the vector characteristics of the data.

[0004] Most existing methods for determining sea surface chlorophyll a concentration utilize multimodal data, failing to explore the common characteristics between different modalities or to segment different physical fields, resulting in data contamination and inevitably failing to achieve good prediction results.

[0005] Therefore, there is an urgent need for a method that can accurately predict the concentration of chlorophyll a on the sea surface in coastal areas, thereby determining the degree of impact of disasters on coastal areas and thus achieving early warning. Summary of the Invention

[0006] This invention provides a method and device for early warning of sea surface chlorophyll a concentration based on multimodal adversarial fusion. By focusing on different physical field information and exploring vectorized information of sea surface wind speed data, this invention improves the accuracy of sea surface chlorophyll a concentration prediction and can provide early warning information of sea surface chlorophyll a concentration to coastal areas 24 hours in advance, thereby preventing the occurrence of harmful red tides and algal blooms. See the description below for details:

[0007] Firstly, a method for early warning of sea surface chlorophyll a concentration based on multimodal adversarial fusion, the method comprising:

[0008] By using block attention to extract the attention between different physical fields, the attention is multiplied by the basic discretized data of the sea surface physical field of each block to obtain the basic representation of the discretized data of the sea surface physical field.

[0009] Using different isobaric surfaces as channels, a convolutional neural network is used to extract basic features from the wind speed component data on each isobaric surface. By utilizing the vector characteristics between the wind speed component data, a specific feature space is constructed, and vector encoding is used to obtain the basic representation of the sea surface wind field data.

[0010] The basic representations of the discretized sea surface physical field data and the sea surface wind field data are input into the adversarial network to achieve multimodal information fusion;

[0011] The fused feature operators are used as objective influencing factors affecting the change of sea surface chlorophyll a concentration. The objective and subjective influencing factors are fused to obtain the feature characterization of the change of sea surface chlorophyll a concentration.

[0012] By utilizing feature characterization to obtain future sea surface chlorophyll a concentration values, we can achieve prediction and early warning of sea surface chlorophyll a concentration.

[0013] In a second aspect, a sea surface chlorophyll a concentration early warning device based on multimodal adversarial fusion, the device comprising: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the device to perform the steps of the method described in any one of the first aspects.

[0014] The beneficial effects of the technical solution provided by this invention are:

[0015] 1. This invention mines the vector characteristics of sea surface wind speed changes and components under different physical fields using two modalities: discretized sea surface physical field data based on expert knowledge and data-driven sea surface wind field data. For discretized sea surface physical field data based on expert knowledge, block attention is used to enhance the interaction between different physical fields. For data-driven sea surface wind field data, a specific feature space is constructed to enhance the correlation between the spatial and vector information of sea surface wind speed components. By integrating multimodal information, a more comprehensive global feature representation can be learned, which helps to better predict and warn of sea surface chlorophyll a concentration.

[0016] 2. This invention utilizes block attention to fully mine the attention intensity of different physical fields under discretized sea surface physical field data based on expert knowledge, and obtains the basic characterization of the two-dimensional structure of physical fields affecting sea surface chlorophyll a concentration; it fully mines the vector characteristics of wind speed component data under data-driven sea surface wind field data, and obtains the basic characterization of sea surface wind field data through vector encoding; it uses adversarial operations to confuse the input multimodal features and reduce distribution differences, thereby further mining the common features between different modes.

[0017] 3. This invention further focuses on different aspects of the multiphysics field of the sea surface, thereby improving the reliability of the multiphysics field feature fusion results; this invention explores the vectorized information of sea surface wind speed in vector space for the first time; finally, this invention fuses multimodal information of the sea surface, improving the accuracy of sea surface chlorophyll a concentration prediction, thereby more accurately preventing the occurrence of harmful red tides and algal blooms in advance. Attached Figure Description

[0018] Figure 1 A flowchart of a sea surface chlorophyll a concentration early warning method based on multimodal adversarial fusion;

[0019] Figure 2 This is a schematic diagram of sea surface chlorophyll a concentration based on multimodal adversarial fusion. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0021] Example 1

[0022] A method for early warning of sea surface chlorophyll a concentration based on multimodal adversarial fusion, see [link to relevant documentation]. Figure 1 The method includes the following steps:

[0023] Step 101: Use the Climate Persistence (CLIPER) method to extract discretized sea surface physical field data, obtain sea surface physical field discretized data based on expert knowledge, and obtain data-driven sea surface wind field data from the sea surface meteorological observation system as input for multimodal data.

[0024] Step 102: For the discretized sea surface physical field data based on expert knowledge, block attention is used to extract the attention between different physical fields. Then, the attention is multiplied by each basic discretized sea surface physical field data to obtain the basic characterization of the discretized sea surface physical field data, thereby enhancing the interaction between different physical fields.

[0025] Step 103: For data-driven sea surface wind field data, different isobaric surfaces are used as different channels, and a convolutional neural network is used to extract basic features from the wind speed component data on each isobaric surface. By utilizing the vector characteristics between the wind speed component data, a specific feature space is constructed. Vector encoding is used to obtain the basic representation of the sea surface wind field data, thereby enhancing the correlation strength between the spatial information and vector information of the sea surface wind speed components.

[0026] Step 104: Input the basic representations of the discretized sea surface physical field data and the basic representations of the sea surface wind field data into the adversarial network. By confusing the data of the two modes, the difference in their sample distribution is reduced, thereby further mining the common characteristics between different modes, realizing multimodal information fusion, and obtaining a feature operator with different physical field correlations and vector characteristics.

[0027] Step 105: The feature operator obtained by fusing the basic characterization of the discretized sea surface physical field data and the basic characterization of the sea surface wind field data is used as the objective influencing factor of the change in sea surface chlorophyll a concentration. The observed trend of sea surface chlorophyll a concentration change 24 hours ago is used as the subjective influencing factor of the change in sea surface chlorophyll a concentration. The two influencing factors are fused to obtain a representative feature characterization of the change in sea surface chlorophyll a concentration.

[0028] Step 106: Using the representative characteristics of sea surface chlorophyll a concentration changes obtained in Step 105, predictions are made to obtain the sea surface chlorophyll a concentration value at future times, thereby achieving prediction and early warning of sea surface chlorophyll a concentration, and thus more accurately preventing the occurrence of harmful red tides and algal blooms in advance.

[0029] In summary, the embodiments of the present invention improve the accuracy of event detection through steps 101-106, meeting various needs in practical applications. The present invention integrates the basic characterization of discretized sea surface physical field data and the basic characterization of sea surface wind field data, focuses on different physical field information and explores the vectorized information of sea surface wind speed data, thereby improving the accuracy of sea surface chlorophyll a concentration prediction and reducing the possibility of harmful red tides and algal blooms.

[0030] Example 2

[0031] The scheme in Example 1 will be further described below with specific calculation formulas and examples:

[0032] 201: The CLIPER method is used to extract discretized sea surface physical field data to obtain sea surface physical field discretized data based on expert knowledge, and data-driven sea surface wind field data is obtained from the sea surface meteorological observation system as input for multimodal data.

[0033] I. Discretized data of sea surface physical fields

[0034] The discretized sea surface physical field data includes: latitude of the sea surface cyclone center (LAT, in degrees), longitude of the sea surface cyclone center (LONG, in degrees), sea surface mean air pressure (PRES, in hectopascals), and sea surface mean wind speed (WND, in m / s). Assuming the future sea surface physical field only possesses 4th-order CLIPER properties, the discretized sea surface physical field data based on expert knowledge can be represented as:

[0035] W = CLIPER(LAT) 0-24 ,LONG 0-24 PRES 0-24 WND 0-24 (1)

[0036] Wherein, W represents the discretized sea surface physical field data based on expert knowledge, which includes 96 characteristic parameters such as latitude and longitude of the sea surface cyclone 24 hours prior, sea surface mean air pressure, sea surface mean wind speed, first-order latitude-longitude difference, first-order wind speed difference, first-order air pressure difference, latitude cosine value, square of sea surface mean wind speed, cube of sea surface mean wind speed, logarithm of sea surface mean wind speed, and ratio of sea surface mean air pressure to sea surface mean wind speed. LAT 0-24 LONG 0-24 PRES 0-24 and WND 0-24 These represent the latitude, longitude, wind speed, and air pressure values ​​for the previous 24 hours, respectively.

[0037] II. Sea Surface Wind Field Data Acquired by the Sea Surface Meteorological Observation System

[0038] A two-dimensional plane of the desired sea area was extracted from the raw sea surface wind field data, and wind speed component data (U and V) were sampled on isobaric surfaces at 250 hPa, 500 hPa, 750 hPa, and 100 hPa to construct a three-dimensional structure of the sea surface cyclone and its surrounding atmospheric circulation in the desired sea area. Finally, a three-dimensional temporal structure of the sea surface cyclone and its surroundings was constructed based on the wind speed component data along four dimensions: longitude, latitude, altitude, and time. This is a data-driven sea surface wind field data model.

[0039] 202: For discretized sea surface physical field data based on expert knowledge, block attention is used to extract the degree of attention between different physical fields. Then, this degree of attention is multiplied by each basic block of discretized sea surface physical field data to obtain the basic characterization of the discretized sea surface physical field data, thereby enhancing the correlation between different physical fields.

[0040] Due to the discretization of sea surface physical field data based on expert knowledge It mainly includes the characteristics of different physical fields. First, it is divided into blocks, each block contains a single physical field, and each physical field contains four characteristic parameters. Therefore, the 96 characteristic parameters can be divided into 24 physical fields.

[0041] To achieve different levels of attention for each physical field, this embodiment of the invention designs a block attention mechanism. Therefore, for the i-th physical field, its attention level can be expressed as:

[0042] q (i) =Q (i) w (i) k (i) =K (i) q (i) ,

[0043]

[0044] Among them, Att (i) Q represents the level of interest in the i-th physical field. (i) and K (i) H1 and H2 are projection matrices, and d is a parameter matrix. out q is the data output dimension of the i-th physical field. (i) For the query obtained by linear operations, k (i) r is the key obtained from the linear operation. (i) The value is the result of a linear operation, T is the matrix transpose, softmax(·) is the normalization exponential function, cat(·) is the dimension concatenation operation, ||·|| is the modulo operation, and w (i) It is the discretized sea surface physical field data based on expert knowledge in the i-th physical field.

[0045] The attention score for each physical field was calculated. By concatenating the attention scores of each physical field, the global attention score, Att, can be obtained. Therefore, the basic representation of the discretized data of sea surface physical fields can be expressed as:

[0046] wide=Att×W (4)

[0047] Where wide represents the basic characterization of the discretized sea surface physical field data, Att is the global level of attention, and W is the discretized sea surface physical field data based on expert knowledge.

[0048] 203: For data-driven sea surface wind field data, different isobaric surfaces are used as different channels, and convolutional neural networks are used to extract basic features from the wind speed component data on each isobaric surface. By utilizing the vector characteristics between wind speed component data, a specific feature space is constructed, and vector encoding operations are used to obtain the basic representation of sea surface wind field data, thereby enhancing the spatial information and vector correlation of sea surface wind speed component information.

[0049] For data-driven sea surface wind field data, which includes two vertical wind speed components (U and V components), this embodiment of the invention first constructs a three-layer convolutional neural network to extract features from these two vertical wind speed components, denoted as:

[0050] X u =SA u ×Conv(U) (5)

[0051] X v =SA v ×Conv(V) (6)

[0052] Where Conv represents a three-layer convolutional neural network, and X u and X v SA represents the feature representations of the U and V components extracted after passing through three layers of a convolutional neural network. u and SA v These represent the spatial attention results for the U and V components, respectively, derived from the ReLU transform and the Sigmoid transform.

[0053] The U and V components are two wind speed components with a vertical relationship. Therefore, this embodiment of the invention constructs a special vector feature space. The U component feature, after passing through three layers of a convolutional neural network, is used as the real part of the vector, and the V component feature, after passing through three layers of a convolutional neural network, is used as the imaginary part of the vector, thus constructing a wind speed vector feature F = X. u +iX vThe wind speed vector features are incorporated into a vector encoding layer, which includes two layers. Since the convolution operator is distributed, the wind speed vector features F are convolved with the vector filtering matrix H to obtain:

[0054] H*F=(A*X u -B*X v )+i(B*X u +A*X v (7)

[0055] Where H = A + iB is a vector filtering matrix. If we use matrix notation to represent the real and imaginary parts of the vector coding layer operations, we can obtain:

[0056]

[0057] in, This represents the real part of the complex number resulting from the vector encoding layer. This represents the imaginary part of the complex number resulting from the vector encoding layer. The result after the vector encoding layer is passed through a complex normalization layer and multiplied by the vector space attention to obtain the basic representation of the final sea surface wind field data.

[0058] deep = SA c ×CReLU(H*F) (9)

[0059] Wherein, deep is the basic representation of sea surface wind field data, CReLU is the vector normalization layer, and SA c This is the result of attention in vector space. Finally, the modulus of deep is calculated, converted to a real number, and then the next steps are performed.

[0060] 204: Input the basic representations of the discretized sea surface physical field data and the basic representations of the sea surface wind field data into the adversarial network. By confusing the data of the two modes, the difference in their sample distribution is reduced, thereby further mining the common characteristics between different modes, realizing multimodal information fusion, and obtaining a feature operator with different physical field correlations and vector characteristics;

[0061] This invention constructs a two-layer linear layer structure as a discriminator for the adversarial module, used to discriminate input features, namely:

[0062] y d =Linear(X) (10)

[0063] Where the input feature X = {wide, deep}, Linear represents a two-layer linear layer structure, y d This represents the structure of the features output by the discriminator. (The y...) dLoss calculation and optimization are performed using the true labels from the category discrimination results. The worse the discriminator's discrimination performance, the better the representation, and the better the data fusion between the two modalities.

[0064] In this embodiment of the invention, the basic representation of the sea surface physical field discretization data (wide) and the basic representation of the sea surface wind field data (deep) are unified into the same dimension and then added proportionally to form the global basic representation G of the sea surface physical field, that is, a feature operator with different physical field correlations and vector characteristics.

[0065] 205: The feature operator after fusing the basic characterization of the discretized sea surface physical field data and the basic characterization of the sea surface wind field data is used as the objective influencing factor of the change in sea surface chlorophyll a concentration. The observed trend of sea surface chlorophyll a concentration change 24 hours ago is used as the subjective influencing factor of the change in sea surface chlorophyll a concentration. The two influencing factors are fused to obtain a representative feature characterization of the change in sea surface chlorophyll a concentration.

[0066] This invention constructs a feature fusion device for subjective and objective influencing factors. In this embodiment, the global basic characterization G of the sea surface physical field obtained in step 204 is used as the objective influencing factor affecting the change in sea surface chlorophyll a concentration, and the observed trend of sea surface chlorophyll a concentration change 24 hours ago is used as C as the subjective influencing factor. Since the direction and angle of the objective influencing factor on sea surface chlorophyll a concentration are not always perpendicular to the sea surface, it is necessary to explore the influence coefficient and direction of the objective influencing factor on sea surface chlorophyll a concentration. Therefore, this embodiment defines a geometric quantity, denoted as:

[0067]

[0068] Where I represents the geometric quantity, which includes the influence coefficient f(G,ω1) of objective influencing factors on sea surface chlorophyll a concentration and the direction of action. This represents the cosine of the angle between the vectors of the sea surface physical field and the trend of chlorophyll a concentration changes. The cosine values ​​of the incident angles, ω1 and ω, represent the effects of the sea surface physical field. λ Let f(·) and f' be parameter matrices. λ f represents a two-layer linear layer. λ It is a refractive index operation on the concentration of chlorophyll a by fitting the sea surface physical field to the cosine value of the incident angle, and ReLU(·) represents the normalization operation.

[0069] To fuse subjective influencing factors with objective influencing factors, this embodiment of the invention also constructs a feature fusion processor, represented as follows:

[0070]

[0071]

[0072] Here, G' represents the characteristic representation of changes in sea surface chlorophyll a concentration, ω0 represents the parameter matrix, f0(·) represents the feature extraction operation on the changing trend of subjective influencing factors, and it is a four-layer LSTM structure. M(t) represents the distribution trend of the effect of the sea surface physical field changing over time, where σ is the standard deviation of the effect of the sea surface physical field changing over time obtained from historical samples during training, and μ is the average value of the effect of the sea surface physical field changing over time obtained from historical samples during training. t (·) represents the operation of obtaining the change in the effect of objective influencing factors over time within a time interval Δt. It is a two-layer MLP layer, where I represents the geometric quantity and Δt represents the change over time. The integral part ∫·dt is an integral over time, aiming to explore the persistent change of sea surface physical fields on sea surface chlorophyll a concentration over a changing time period. S 2 k represents the area of ​​the sea surface affected. d and k s The parameter represents the learnable parameter, and softmax(·) represents the normalization operation.

[0073] Step 206: Using the representative feature characterization G' of sea surface chlorophyll a concentration change obtained in step 205, prediction is made to obtain the sea surface chlorophyll a concentration value at future times, thereby realizing the prediction and early warning of sea surface chlorophyll a concentration.

[0074] This invention constructs a two-layer linear layer as a classifier, namely:

[0075]

[0076] in, This represents the predicted sea surface chlorophyll a concentration, and `classify` represents the classifier (two linear layers). The predicted sea surface chlorophyll a concentration results are then... Calculate the loss using the true label y:

[0077]

[0078] In this embodiment of the invention, the predicted sea surface chlorophyll a concentration is classified into levels according to the concentration and displayed with different colors. For example, a concentration greater than 10 ug / L corresponds to red, indicating that the water body may be eutrophic, while a concentration less than 10 ug / L corresponds to blue, indicating that the water body is not eutrophic.

[0079] Based on different risk levels, different colors are used to provide advance notice and warnings to professionals, as well as to enable them to respond to emergencies and ensure the stability and safety of the marine environment in coastal areas.

[0080] Example 3

[0081] The feasibility of Examples 1 and 2 is verified through specific experiments, as detailed below:

[0082] For example, this invention selected sea surface physical field (SMP) data from 2000 to 2014 as the training set and sea surface SMP data from 2015 to 2018 as the test set. The sea surface chlorophyll a concentration prediction method of this embodiment uses discretized sea surface SMP data and sea surface wind field data from the previous 24 hours to predict the sea surface chlorophyll a concentration 24 hours later. First, the discretized sea surface SMP data is divided into blocks, and the attention level for each block is obtained to acquire the basic characterization of the discretized sea surface SMP data. Then, a specific vector space is constructed for the sea surface wind field data, and vector encoding is used to obtain the basic characterization of the sea surface wind field data. Furthermore, an adversarial module is constructed to obfuscate the data from two modalities, reducing the difference in their sample distribution, thereby further mining the common characteristics between different modalities and achieving multimodal information fusion. Finally, the obtained objective and subjective influencing factors are fused using a feature fusion processor to obtain a representative feature characterization of sea surface chlorophyll a concentration.

[0083] This invention combines the basic representations of discretized sea surface physical field data and sea surface wind field data to form a global basic representation of the sea surface physical field. This representation is then fused with the trend characteristics of sea surface chlorophyll a concentration changes, and finally, sea surface chlorophyll a concentration is predicted. This invention also uses the Mean Absolute Error (MAE) index to evaluate the performance of the method. MAE represents the mean absolute error between the predicted intensity and the observed intensity. A smaller MAE error indicates a more accurate prediction. This invention provides a more reliable method for predicting and warning of sea surface chlorophyll a concentration, and it achieves different focuses on multiple physical fields, further exploring the structured characteristics of sea surface wind speed from the vector space.

[0084] Example 4

[0085] A sea surface chlorophyll a concentration early warning device based on multimodal adversarial fusion, the device comprising: a processor and a memory, wherein the memory stores program instructions, and the processor calls the program instructions stored in the memory to cause the device to perform the following method steps:

[0086] By using block attention to extract the attention between different physical fields, the attention is multiplied by the basic discretized data of the sea surface physical field of each block to obtain the basic representation of the discretized data of the sea surface physical field.

[0087] Using different isobaric surfaces as channels, a convolutional neural network is used to extract basic features from the wind speed component data on each isobaric surface. By utilizing the vector characteristics between the wind speed component data, a specific feature space is constructed, and vector encoding is used to obtain the basic representation of the sea surface wind field data.

[0088] The basic representations of the discretized sea surface physical field data and the sea surface wind field data are input into the adversarial network to achieve multimodal information fusion;

[0089] The fused feature operators are used as objective influencing factors affecting the change of sea surface chlorophyll a concentration. The objective and subjective influencing factors are fused to obtain the feature characterization of the change of sea surface chlorophyll a concentration.

[0090] By utilizing feature characterization to obtain future sea surface chlorophyll a concentration values, we can achieve prediction and early warning of sea surface chlorophyll a concentration.

[0091] Specifically, the extraction of attention between different physical fields using block attention is as follows:

[0092] q (i) =Q (i) w (i) k (i) =K (i) q (i) ,

[0093]

[0094] Among them, Att (i) Q represents the level of interest in the i-th physical field. (i) and K (i) H1 and H2 are projection matrices, and d is a parameter matrix. out q is the data output dimension of the i-th physical field. (i) For the query obtained by linear operations, k (i) r is the key obtained from the linear operation. (i) The value is the result of a linear operation, T is the matrix transpose, softmax(·) is the normalization exponential function, cat(·) is the dimension concatenation operation, ||·|| is the modulo operation, and w (i) It is the discretized sea surface physical field data based on expert knowledge in the i-th physical field.

[0095] The basic characterization of the discretized data of the sea surface physical field is as follows:

[0096] wide = Att × W

[0097] Where wide represents the basic characterization of the discretized sea surface physical field data, Att is the global level of attention, and W is the discretized sea surface physical field data based on expert knowledge.

[0098] Specifically, by utilizing the vector characteristics between wind speed component data to construct a specific feature space, and using vector encoding to obtain the basic representation of sea surface wind field data, the following steps are taken:

[0099] The U-component features after passing through three layers of convolutional neural network are used as the real part of the vector, and the V-component features after passing through three layers of convolutional neural network are used as the imaginary part of the vector, thus constructing a wind speed vector feature F = X. u +iX v The wind speed vector features are put into a vector encoding, which contains a two-layer vector encoding layer. The wind speed vector features F are convolved through the vector filtering matrix H.

[0100] The result after passing through the vector encoding layer is passed through the complex normalization layer and multiplied with the vector space attention to obtain the basic representation of the final sea surface wind field data.

[0101] Furthermore, the method also includes:

[0102] A two-layer linear layer structure is constructed as the discriminator of the adversarial module to discriminate the input features; the features are processed by the structure output by the discriminator and the true labels of the class discrimination results to calculate and optimize the loss.

[0103] The basic representations of the sea surface physical field discretization data (wide) and the basic representations of the sea surface wind field data (deep) are unified into the same dimension and then added proportionally to form the global basic representation G of the sea surface physical field.

[0104] Furthermore, the method also includes: defining a geometric quantity:

[0105]

[0106] Where I represents the geometric quantity, which includes the influence coefficient f(G,ω1) of objective influencing factors on sea surface chlorophyll a concentration and the direction of action. ω1 and ω λ Let f(·) and f' be parameter matrices. λ f represents a two-layer linear layer. λ It is a refractive index operation on the concentration of chlorophyll a by fitting the sea surface physical field to the cosine value of the incident angle, and ReLU(·) represents the normalization operation.

[0107] The feature fusion of objective and subjective influencing factors is performed as follows: a feature fusion engine is constructed, represented as:

[0108]

[0109]

[0110] Where G' represents the characteristic representation of the change in sea surface chlorophyll a concentration, ω0 represents the parameter matrix, f0(·) represents the feature extraction operation on the changing trend of subjective influencing factors, M(t) represents the distribution trend of the effect of the sea surface physical field over time, where σ is the standard deviation, μ is the average effect, and f t (·) represents the change in effect, and is a two-layer MLP layer, S 2 k represents the area of ​​the sea surface affected. d and k s This represents the learnable parameters.

[0111] It should be noted that the device descriptions in the above embodiments correspond to the method descriptions in the embodiments, and the embodiments of the present invention will not be repeated here.

[0112] The execution entities of the aforementioned processor and memory can be devices with computing functions such as computers, microcontrollers, and single-chip microcomputers. In specific implementations, the embodiments of the present invention do not limit the execution entities and can select them according to the needs of actual applications.

[0113] Data signals are transmitted between the memory and the processor via a bus, which will not be elaborated upon in this embodiment of the invention.

[0114] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium, the storage medium including a stored program, which, when the program is running, controls the device where the storage medium is located to execute the method steps in the above embodiments.

[0115] The computer-readable storage medium includes, but is not limited to, flash memory, hard disk, solid-state drive, etc.

[0116] It should be noted that the description of the readable storage medium in the above embodiments corresponds to the description of the method in the embodiments, and the embodiments of the present invention will not be repeated here.

[0117] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated.

[0118] A computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in or transmitted through a computer-readable storage medium. A computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic or semiconductor, etc.

[0119] References

[0120] [1] Zhang Kexin, Lu Kaihong, Jin Chunhua, et al. Study on chlorophyll a concentration prediction model based on BP neural network in Zhenshui Reservoir, Hunan Province [J]. Bulletin of Marine Limnology and Geophysics, 2011(2):91-99. DOI:10.3969 / j.issn.1003-6482.2011.02.013.

[0121] [2] Hu Zhiyang, Li Cuimei, Xue Tianyi. Chlorophyll a concentration prediction based on grey relational analysis and ABC-BP neural network [J]. Hydropower Energy Science, 2021, 39(1):55-58.

[0122] [3] Shi Suixiang, Wang Lei, Yu Xuan, et al. Application of long short-term memory neural network in chlorophyll a concentration prediction [J]. Acta Oceanologica Sinica (Chinese Edition), 2020, 42(2):134-142. DOI:10.3969 / j.issn.0253-4193.2020.02.014.

[0123] [4] Yi Yang, He Xianbo, Wang Chunrui. Study on short-term prediction of chlorophyll a concentration based on singular spectrum analysis and long short-term memory neural network [J]. Intelligent Computer and Applications, 2022, 12(11):134-137. DOI:10.3969 / j.issn.2095-2163.2022.11.021.

[0124] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for early warning of sea surface chlorophyll-a concentration based on multi-modal adversarial fusion, characterized in that, The method includes: By using block attention to extract the attention between different physical fields, the attention is multiplied by the basic discretized data of the sea surface physical field of each block to obtain the basic representation of the discretized data of the sea surface physical field. Using different isobaric surfaces as channels, a convolutional neural network is used to extract basic features from the wind speed component data on each isobaric surface. By utilizing the vector characteristics between the wind speed component data, a specific feature space is constructed, and vector encoding is used to obtain the basic representation of the sea surface wind field data. The basic representations of the discretized sea surface physical field data and the sea surface wind field data are input into the adversarial network to achieve multimodal information fusion; The fused feature operators are used as objective influencing factors affecting the change of sea surface chlorophyll a concentration. The objective and subjective influencing factors are fused to obtain the feature characterization of the change of sea surface chlorophyll a concentration. By utilizing feature characterization to obtain future sea surface chlorophyll a concentration values, we can achieve prediction and early warning of sea surface chlorophyll a concentration. Specifically, the method of constructing a specific feature space by utilizing the vector characteristics between wind speed component data and obtaining the basic representation of sea surface wind field data using vector encoding is as follows: The U-component features after passing through three convolutional neural network layers are used as the real part of the vector, and the V-component features after passing through three convolutional neural network layers are used as the imaginary part of the vector to construct a wind speed vector feature. The wind speed vector features are incorporated into a vector encoding process, which includes a two-layer vector encoding layer. Through vector filtering matrix convolution; The result after passing through the vector encoding layer is passed through the complex normalization layer and multiplied with the vector space attention to obtain the basic representation of the final sea surface wind field data.

2. The sea surface chlorophyll a concentration early warning method based on multimodal adversarial fusion according to claim 1, characterized in that, The specific method of extracting the attention between different physical fields using block attention is as follows: ； ； in, Indicates for the first The level of attention to a physical field and It is a projection matrix. and It is a parameter matrix. It is the first Each physical field's data output dimension The query obtained from linear operations. The key obtained from linear operations. The value is the result of a linear operation, and T is the matrix transpose operation. It is a normalized exponential function. It's a dimension splicing operation. It is a modulo operation. It is the first Discretized sea surface physics data based on expert knowledge in a physical field.

3. The sea surface chlorophyll a concentration early warning method based on multimodal adversarial fusion according to claim 1, characterized in that, The basic characterization of the discretized sea surface physical field data is as follows: ； in, This represents the basic characterization of discretized data of the sea surface physical field. It's about overall attention. It is discretized data of the sea surface physical field based on expert knowledge.

4. The sea surface chlorophyll a concentration early warning method based on multimodal adversarial fusion according to claim 1, characterized in that, The method further includes: A two-layer linear layer structure is constructed as the discriminator of the adversarial module to discriminate the input features; the features are processed by the structure output by the discriminator and the true labels of the class discrimination results to calculate and optimize the loss. Basic characterization of discretized sea surface physical field data Basic characterization of sea surface wind field data After unifying to the same dimension, they are added proportionally to serve as the global fundamental characterization of the sea surface physical field. .

5. The sea surface chlorophyll a concentration early warning method based on multimodal adversarial fusion according to claim 4, characterized in that, The method further includes: defining a geometric quantity: ； in, This represents a geometric quantity, which includes the influence coefficients of objective influencing factors on sea surface chlorophyll a concentration. and direction of action , and Represents the parameter matrix, and Represents a two-layer linear layer The refractive index manipulation of chlorophyll a concentration is achieved by fitting the cosine value of the incident angle to the sea surface physical field. This indicates a normalization operation, where C represents the trend of sea surface chlorophyll a concentration changes 24 hours prior.

6. The sea surface chlorophyll a concentration early warning method based on multimodal adversarial fusion according to claim 5, characterized in that, The feature fusion of objective and subjective influencing factors is described as follows: A feature fusion engine is constructed, represented as: ； ； in, Characteristic representation of changes in sea surface chlorophyll a concentration. Represents the parameter matrix, This represents a feature extraction operation that describes the changing trends of subjective influencing factors. This indicates the distribution trend of the effects of the sea surface physical field over time, where It is the standard deviation. It is the average value of the effect. The operation representing the change in effect is a two-layer MLP. Indicates the area of ​​the sea surface affected. and This represents the learnable parameters.

7. A sea surface chlorophyll a concentration early warning device based on multimodal adversarial fusion, characterized in that, The device includes a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the device to perform the steps of the method according to any one of claims 1-6.

Images