A Multimodal Algal Bloom Recognition and Prediction Method Based on Graph Attention Aggregation

Abstract

This invention presents a multimodal algal bloom recognition and prediction method based on graph attention aggregation, belonging to the interdisciplinary fields of intelligent ecological environment monitoring and artificial intelligence. The method collects multimodal data within a continuous time window, performs spatial alignment and sliding window partitioning; constructs a dynamic cross-modal graph within each window; extracts the global static embedding and local temporal embedding of nodes, aggregates neighbor information through an attention mechanism, updates node temporal embeddings, and calculates a dynamic relationship matrix; extracts and fuses global temporal features using a temporal convolutional network; and finally outputs the algal bloom probability distribution to generate an early warning map. This invention integrates multimodal data, introduces dual-embedding dynamic graph attention and sliding window temporal modeling, overcoming the shortcomings of existing technologies such as single modality, static graph structure, and lack of temporal prediction, significantly improving the accuracy and reliability of algal bloom recognition and prediction.

Description

Technical Field

[0001] This invention relates to a method for identifying and predicting multimodal algal blooms based on graph attention aggregation, belonging to the interdisciplinary fields of intelligent ecological environment monitoring and artificial intelligence, and is particularly applicable to the identification and prediction of multimodal algal blooms based on graph attention aggregation. Background Technology

[0002] Cyanobacterial blooms deplete dissolved oxygen and release algal toxins, seriously threatening drinking water safety and ecological balance. With the increasing prominence of cyanobacterial blooms globally, achieving high-precision identification and short-term prediction of blooms has become a crucial issue in the field of ecological and environmental monitoring.

[0003] Existing methods for monitoring algal blooms mainly include on-site sampling and analysis, remote sensing monitoring, and deep learning methods. On-site sampling has high accuracy but limited range and poor timeliness; remote sensing monitoring can cover a large area but is easily affected by cloud cover and only reflects information from the water surface; deep learning methods such as convolutional neural networks can extract spectral-spatial features, but it is difficult to explicitly model the spatial diffusion relationship of algal blooms.

[0004] Graph neural networks (Graph Neural Networks) can naturally handle non-Euclidean structured data and have been introduced into the field of remote sensing image classification in recent years. Patent CN119888367A discloses a method and apparatus for classifying hyperspectral remote sensing images based on graph neural networks. This method constructs an adjacency matrix by treating each pixel in the hyperspectral remote sensing image as a node. It uses a cosine similarity metric function to identify and remove abnormal pixels (i.e., pixel pairs with negative cosine similarity) from first-order neighbor nodes, obtaining an optimized adjacency matrix. Then, it aggregates the neighbor features of each pixel using an average aggregation method and extracts the features of the aggregated center pixel using a KAN network, ultimately achieving classification. This method effectively improves the classification efficiency and accuracy of hyperspectral remote sensing images. Patent CN120071139A discloses a method and system for detecting red tides in hyperspectral remote sensing images. This method converts the hyperspectral remote sensing image into a hypergraph structure composed of vertices and hyperedges, uses a hypergraph neural network for feature mapping, and designs an unsupervised loss function that requires no annotation. This loss function ensures that vertices with similar spectra remain spatially close, thereby achieving clustering and differentiation between red tides and seawater.

[0005] However, existing methods have the following common technical challenges: (1) They are single-modal and mostly rely on remote sensing images, without effectively integrating multi-source data such as meteorological and sensor data; (2) The graph structure is based on spatial adjacency and cannot capture long-distance dependencies between algal bloom regions that are not spatially adjacent but have similar spectral characteristics; (3) They lack the ability to model time series and can only classify the current time, and cannot achieve short-term prediction of the spatial distribution of algal blooms. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a multimodal algal bloom recognition and prediction method based on graph attention aggregation. By fusing remote sensing images, water surface images, meteorological images and sensor data, a single-modal or cross-modal graph structure is constructed. At the same time, sliding window temporal modeling and dual-embedded dynamic graph attention aggregation network are introduced to achieve short-term prediction of the spatial distribution of algal blooms.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A multimodal algal bloom recognition and prediction method based on graph attention aggregation is characterized by comprising the following steps:

[0009] S1: Collect multimodal data of the monitoring area within a continuous time window, and perform spatial alignment, time series construction, and sliding window division on the collected multimodal data;

[0010] S2: Within each sliding window, a graph structure is independently constructed based on spatial adjacency and cross-modal similarity thresholds to obtain a dynamic cross-modal graph;

[0011] S3: Extract the global static embedding and the local temporal embedding based on the sliding window for each node in the dynamic cross-modal graph;

[0012] S4: After concatenating the global static embedding and the local temporal embedding, the attention mechanism is used to aggregate neighbor information on the graph structure, update the node temporal embedding, and calculate the dynamic relationship matrix;

[0013] S5: Use a one-dimensional temporal convolutional network (TCN) to extract global temporal features from multimodal data and fuse them with the dynamic relation matrix to obtain fused features;

[0014] S6: The fused features are analyzed through a fully connected layer and the Softmax function to output the probability distribution of algal bloom at each node at the predicted time. After binarization, an algal bloom warning map is generated.

[0015] Specifically, the graph structure involves dividing the monitoring area into... Each spatial unit is considered as a node, representing a multimodal feature vector of a geographic location. The undirected line connecting two nodes is an edge, indicating that there is a spatial adjacency or feature similarity relationship between the two spatial units. The edge weight reflects their similarity.

[0016] Preferably, the size of the regular spatial unit is selected reasonably according to the needs of the scene, which depends on the computational efficiency and resolution; the smaller the regular spatial unit, the worse the computational efficiency and the higher the resolution, and vice versa.

[0017] Furthermore, the multimodal data includes, but is not limited to, multi-source remote sensing images. Water surface images Meteorological raster data and sensor data One or more of the following; among which, For the time of data collection, , For the image height and width; This represents the number of spectral bands. This refers to the number of meteorological variables, including temperature, wind speed, solar radiation, and precipitation. Number of sensor sites; This refers to the number of water quality indicators, including water temperature, pH, dissolved oxygen, chlorophyll a concentration, and phycocyanin concentration.

[0018] Furthermore, step S1 specifically includes:

[0019] S101: Collect multimodal data of the monitoring area within a continuous time window;

[0020] S102: Divide the monitoring area into Each spatial unit is a regular spatial unit, and spatial interpolation and alignment are performed on all modal data so that each spatial unit obtains a complete feature vector at each time step.

[0021] S103: Let the time window length be... For any time Constructing time series:

[0022] ;

[0023] in For a moment A multimodal data set, For the total number of modes, For modality At any moment The data matrix of the entire monitoring area, For modality Data dimensions;

[0024] S104: Convert the time series Divide into equal lengths based on the time length rounded down. There are 10 sliding sub-windows; each sliding sub-window has a length of 10 ... , This represents the floor division method, the first floor division method. The sliding sub-window is , For the first The start time of each sliding child window .

[0025] Furthermore, step S2 specifically includes:

[0026] S201: For any sliding child window and modality Calculate the time average of the features of each node within the window. ,in For nodes In modality Next moment eigenvectors, ;

[0027] S202: For each mode Calculate any two nodes , Feature cosine similarity between If node and Spatially adjacent and Then, construct a modal interior edge with an edge weight set to . ;in For modality The similarity threshold; where, It is a 2-norm. For transpose;

[0028] S203: For any two different modes and Calculate two nodes , Cross-modal similarity between ,like Then, an inter-modal edge is established, with the edge weight set to . ;in The cross-modal similarity threshold;

[0029] S204: Construct a sliding sub-window based on the union of modal inner edges and inter-modal edges. The dynamic cross-modal graph; specifically,

[0030] (1) For two nodes , Take the maximum weight value from the union of the edges: ;

[0031] (2) Construct the adjacency matrix ,in And add an identity diagonal matrix get Then, symmetric normalization is performed to obtain the comprehensive adjacency matrix. ;in, For degree matrix, .

[0032] Furthermore, step S3 specifically includes:

[0033] S301: For each node Assign a learnable global static embedding vector , representing inherent properties that do not change over time, are optimized through backpropagation during network training; among them, A positive integer that is set by the user;

[0034] S302: For each sliding sub-window and each mode Nodes are extracted using a one-dimensional standard temporal convolutional network (1D-CNN). Temporal dynamic characteristics The kernel size is... , For modality No. The weight matrix of each convolutional kernel. For bias terms, This is a characteristic sequence.

[0035] S303: For all The temporal dynamic features of each modality are averaged and fused to obtain the node. In the sliding sub-window Final time embedding Then, the local time embedding matrix is ​​obtained by combining all nodes. .

[0036] Preferably, select .

[0037] Furthermore, step S4 specifically includes:

[0038] S401: Concatenate the global static embedding and the local temporal embedding to obtain a fused dual embedding vector. ;in This represents the operation of concatenating the beginning and end of a vector along its dimension.

[0039] S402: Compute Node To his neighbors Unnormalized attention score:

[0040] ;

[0041] Among them, nodes Neighbor set , This is a learnable attention parameter vector. , Hyperparameters are set manually;

[0042] The attention weights are obtained by normalizing the neighbor nodes using the Softmax function.

[0043] ;

[0044] satisfy and ;

[0045] S403: Use attention weights to perform weighted aggregation of the local temporal embeddings of neighboring nodes, and then pass them through an activation function to obtain the updated temporal embeddings. This leads to the updated time embedding matrix. ;in , For hyperparameters;

[0046] S404: Based on the updated temporal embedding, calculate the cosine similarity matrix between nodes as the local dynamic relationship matrix. ;in, ;

[0047] S405: Adopted Each attention head independently calculates steps S402~S404 above, resulting in... The updated time embedding matrix And concatenate the multi-head results along the feature dimension: The corresponding local dynamic relation matrix is ​​the average value of each head: ,in, This is for splicing operations.

[0048] Furthermore, step S5 specifically includes:

[0049] S501: Extract global temporal features from multimodal time series within a time window using a one-dimensional temporal convolutional network. ;in, , For modality, Number of output channels It is a one-dimensional temporal convolutional layer. ;

[0050] S502: Global time series characteristics Divided by sliding window A fragment , Then, each global temporal feature segment is reshaped into a matrix. , and the local dynamic relationship matrix Perform matrix multiplication Finally, the fusion features of all windows are concatenated along the feature dimensions to form the fusion feature. ; This represents the number of sliding child windows.

[0051] Furthermore, step S6 specifically includes:

[0052] S601: A node-level fully connected layer is used to output the probability that each node belongs to an algal bloom, resulting in a probability matrix. ,in , , , For nodes Region in forecast time The probability of algal bloom occurring at any given time , To predict the step size;

[0053] S602: Threshold After binarization, a spatial distribution prediction map of algal blooms is obtained; where the corresponding nodes... The regional algal bloom classification binary value ;

[0054] S603: Integrate the prediction results with the geographic information system to generate an algal bloom early warning map and mark high-risk areas. When the predicted algal bloom area exceeds the set threshold, an early warning signal is automatically triggered, notifying the environmental monitoring department to take corresponding prevention and control measures.

[0055] Preferably, for the prediction of whether algal blooms will occur in the overall monitored water area, a learnable method is used before step S6. A convolutional layer followed by a ReLU layer, and then a max-pooling layer, are used to downsample the fused features. ,Then Perform global average pooling at the node level and update the fused features. ;in, for convolution; This is a max pooling operation.

[0056] Furthermore, the network parameters are trained end-to-end using cross-entropy loss and the Adam optimizer. The cross-entropy loss function is: ,in For nodes The true label at the predicted moment.

[0057] Preferably, one or more prediction step sizes can be selected based on the specific needs of the scenario. And train the corresponding network model, when The current time represents the probability of algal bloom. The predicted probability of algal blooms at a future time.

[0058] An electronic device includes at least one processor; and a memory communicatively connected to said at least one processor; wherein,

[0059] The memory stores a computer program that is executed by the at least one processor, which enables the at least one processor to perform the multimodal algal bloom recognition and prediction method based on graph attention aggregation described above.

[0060] Finally, the present invention also discloses a computer-readable storage medium storing computer instructions for causing a processor to execute the above-described method for identifying and predicting multimodal algal blooms based on graph attention aggregation.

[0061] The beneficial effects of this invention are as follows: It provides a multimodal algal bloom recognition and prediction method based on graph attention aggregation. By constructing a multimodal cross-modal graph structure, it realizes long-distance semantic information transmission between different modalities. At the same time, it introduces dynamic graph attention aggregation with dual embedding of global static embedding and local temporal embedding to realize joint modeling of global static attributes and local temporal dynamics. Combined with sliding window temporal convolution, it realizes end-to-end prediction from historical observation to future spatial distribution of algal blooms. This overcomes the defects of existing technologies such as single modality, static graph structure and lack of temporal prediction ability, and significantly improves the accuracy of algal bloom recognition and the reliability of short-term prediction. Attached Figure Description

[0062] To make the objectives and technical solutions of this invention clearer, the following figures are provided for illustration:

[0063] Figure 1 This is a flowchart of the multimodal algal bloom recognition and prediction method based on graph attention aggregation in this invention;

[0064] Figure 2 This is a schematic diagram of cross-modal graph construction in Embodiment 1 of the present invention;

[0065] Figure 3 This is a schematic diagram of the dual-embedded dynamic graph attention aggregation network architecture in Embodiment 1 of the present invention;

[0066] Figure 4 This is a flowchart of the single-modal algal bloom recognition and prediction method based on graph attention aggregation in Embodiment 2 of the present invention.

[0067] Figure 5 This is a schematic diagram of the electronic device in Embodiment 4 of the present invention. Detailed Implementation

[0068] To make the objectives and technical solutions of this invention clearer, the invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0069] Example 1: Due to eutrophication, Lake A has long experienced frequent cyanobacterial blooms, which have previously caused drinking water crises, affecting the water supply of millions of residents. This example uses a comprehensive time-series dataset related to cyanobacterial blooms in Lake A to conduct a short-term prediction experiment for these blooms.

[0070] This dataset, constructed based on field measurements and satellite remote sensing technology, contains 26 variables categorized into four types: water quality, bio-optics, climate, and human activities. It spans over 15 years, with the satellite data exhibiting a spatial resolution ranging from 30 m to 500 m. This embodiment uses data from 2020 to 2023, a total of four years, for experimental purposes.

[0071] Therefore, this invention provides a multimodal algal bloom recognition and prediction method based on graph attention aggregation, combining... Figure 1 It includes the following steps:

[0072] Step S1: Collect multimodal data of the monitoring area within a continuous time window, and perform spatial alignment, time series construction, and sliding window division on the collected multimodal data. Specifically,

[0073] S101: Collect three modal data for Lake A region from January 2020 to December 2023, a total of four years, including:

[0074] • Remote sensing image data: satellite imagery with a spatial resolution of 250 m and a temporal resolution of 1 day. Bands B1 (620-670 nm, red), B2 (841-876 nm, near-infrared), B3 (459-479 nm, blue), and B4 (545-565 nm, green) were selected to calculate algal bloom sensitivity indices such as NDVI and FAI.

[0075] • Meteorological raster data: Reanalysis dataset containing daily average temperature (°C), daily average wind speed (m / s), daily total solar radiation (W / m²), and daily precipitation (mm / h), with a spatial resolution of 0.25°×0.25° (approximately 28 km) and interpolated to a resolution of 250 m.

[0076] • Sensor data: Daily water quality data from 11 buoy stations in Lake A, including water temperature (°C), pH, dissolved oxygen (mg / L), and chlorophyll a concentration (μg / L).

[0077] S102: Divide the monitoring area into There are 10 regular spatial units, each unit approximately 10 ... And perform spatial interpolation and alignment on all modal data;

[0078] S103: Let the time window length be... (Corresponding to 10 days), predicted step size (Predicting 1 day from now), for any given time Constructing time series:

[0079]

[0080] in For a moment A multimodal data set, For the total number of modes, For modality At any moment The data matrix of the entire monitoring area, For modality Data dimensions ;

[0081] S104: Convert the time series Divide into equal lengths based on the time length rounded down. There are 10 sliding sub-windows; each sliding sub-window has a length of 10 ... , This represents the floor division method, the first floor division method. The sliding sub-window is , , , , .

[0082] Step S2: Combining Figure 2 Within each sliding window, a graph structure is independently constructed based on spatial adjacency and cross-modal similarity thresholds to obtain a dynamic cross-modal graph. Specifically,

[0083] S201: For any sliding child window and modality Calculate the time average of the features of each node within the window. ,in For nodes In modality Next moment eigenvectors, ;

[0084] S202: For each mode Calculate any two nodes , Feature cosine similarity between If node and Spatially adjacent and Then, construct a modal internal edge that requires spatial adjacency, with the edge weight set to . ;in For modality The similarity threshold; where, It is a 2-norm. For transpose;

[0085] S203: For any two different modes and Calculate two nodes , Cross-modal similarity between ,like Then, an intermodal edge is established, which does not require spatial adjacency, and the edge weight is set to . ;in The cross-modal similarity threshold;

[0086] S204: Construct a sliding sub-window based on the union of modal inner edges and inter-modal edges. The dynamic cross-modal graph; specifically,

[0087] (1) For two nodes , Take the maximum weight value from the union of the edges: ;

[0088] (2) Construct the adjacency matrix ,in And add an identity diagonal matrix get Then, symmetric normalization is performed to obtain the comprehensive adjacency matrix. ;in, For degree matrix, .

[0089] Step S3: Combining Figure 3 Extract the global static embedding and the sliding window-based local temporal embedding of each node in the dynamic cross-modal graph. Specifically,

[0090] S301: For each node Assign a learnable global static embedding vector , and randomly initialized, representing inherent properties that do not change over time, which are optimized through backpropagation during network training; among them, ;

[0091] S302: For each sliding sub-window and each mode Node extraction using 1D-CNN Temporal dynamic characteristics The kernel size is... , For modality No. The weight matrix of each convolutional kernel. For bias terms, This is a characteristic sequence.

[0092] S303: Average and fuse the temporal dynamic features of all three modalities to obtain the node. In the sliding sub-window Final time embedding Then, the local time embedding matrix is ​​obtained by combining all nodes. .

[0093] Step S4: After concatenating the global static embedding and the local temporal embedding, neighbor information is aggregated on the graph structure using an attention mechanism, the node temporal embedding is updated, and the dynamic relation matrix is ​​calculated. Specifically,

[0094] S401: Concatenate the global static embedding and the local temporal embedding to obtain a fused dual embedding vector. ;in This represents the operation of concatenating the beginning and end of a vector along its dimension.

[0095] S402: Compute Node To his neighbors Unnormalized attention score:

[0096] ;

[0097] Among them, nodes Neighbor set , This is a learnable attention parameter vector. negative slope ;

[0098] The attention weights are obtained by normalizing the neighbor nodes using the Softmax function.

[0099] ;

[0100] satisfy and ;

[0101] S403: Use attention weights to perform weighted aggregation of the local temporal embeddings of neighboring nodes, and then pass them through an activation function to obtain the updated temporal embeddings. This leads to the updated time embedding matrix. ;in , For hyperparameters;

[0102] S404: Based on the updated temporal embedding, calculate the cosine similarity matrix between nodes as the local dynamic relationship matrix. ;in, ;

[0103] S405: Adopted Each attention head independently calculates steps S402~S404 above, resulting in... The updated time embedding matrix And concatenate the multi-head results along the feature dimension: The corresponding local dynamic relation matrix is ​​the average value of each head: ,in, This is for splicing operations.

[0104] Step S5: Extract global temporal features from the multimodal data using TCN and fuse them with the dynamic relation matrix to obtain fused features. Specifically,

[0105] S501: Use TCN to extract global temporal features from multimodal time series within a time window. ;in, ; ; Modal; Number of output channels; It is a TCN, which contains 3 layers of dilated convolutions with a kernel size of 3 and dilation coefficients of 1, 2 and 4 respectively;

[0106] S502: Will Divided by sliding window A fragment , Then, each segment of the global temporal features is reshaped into a matrix. , and the local dynamic relationship matrix Perform matrix multiplication Finally, the fusion features of all windows are concatenated along the feature dimensions to form the fusion feature. .

[0107] Step S6: Fuse features using a fully connected layer and a softmax function. The analysis outputs the probability distribution of algal blooms at each node at the predicted time, which is then binarized to generate an algal bloom early warning map. Specifically,

[0108] S601: A node-level fully connected layer is used to output the probability that each node belongs to an algal bloom, resulting in a probability matrix. ,in The weight matrix to be trained. Let be the bias vector to be trained. For nodes Region in forecast time The probability of algal bloom occurring at any given time , To predict the step size;

[0109] S602: Threshold After binarization, a spatial distribution prediction map of algal blooms is obtained; where the corresponding nodes... The regional algal bloom classification binary value ;

[0110] S603: Integrate the prediction results with the geographic information system to generate an algal bloom early warning map and mark high-risk areas. When the predicted algal bloom area exceeds the set threshold, an early warning signal is automatically triggered, notifying the environmental monitoring department to take corresponding prevention and control measures.

[0111] In this embodiment, the Adam optimizer is used to update network parameters. An initial learning rate is set. First-order moment attenuation coefficient Second-order moment attenuation coefficient Numerical stability factor Batch size The loss function is node-level cross-entropy loss: An early stopping strategy was adopted: training was stopped when the validation set loss did not decrease for 20 consecutive rounds, with a maximum training round count of 200. The training set used data from 2020 to 2022 (approximately 1095 days), the validation set used data from January to June 2023, and the test set used data from July to December 2023.

[0112] Example 2: Rapid screening scenario for which only a single high-resolution remote sensing image is available and temporal prediction is not required. Spatial identification of cyanobacterial blooms in Lake A is performed using 10-meter resolution multispectral imagery from the Sentinel-2 satellite. The Sentinel-2 satellite provides RGB and near-infrared bands with a spatial resolution of 10 meters and a revisit period of 5 days. This example selects a Sentinel-2 L2A atmospheric-corrected image of the Lake A region taken on August 15, 2023. The cloud cover in this image is less than 10%, and ground features are clearly visible.

[0113] The provided Sentinel2 10-meter resolution cyanobacterial bloom dataset for Lake A contains cyanobacterial bloom distribution data at multiple time points, and the cyanobacterial bloom distribution data around August 15, 2023 can be used as the verification benchmark for this embodiment.

[0114] Therefore, this invention provides a method for identifying and predicting single-modal algal blooms based on graph attention aggregation, combined with... Figure 5 It includes the following steps:

[0115] Step S1: Acquisition and preprocessing of single-modal time series data.

[0116] This embodiment uses only a single Sentinel-2 remote sensing image. The Lake A region is divided into... Each spatial unit corresponds to one pixel (10 m × 10 m). For a single image, the history window length is... Number of sliding windows It does not involve temporal modeling. The feature vector of each pixel node... It includes four bands of Sentinel-2: B2 (blue, 490nm), B3 (green, 560nm), B4 (red, 665nm), and B8 (near-infrared, 842nm).

[0117] Step S2: Construction of single-modal static graph structure.

[0118] For modes (Remote sensing modality), modal inner edges: calculate the cosine similarity of spatially adjacent nodes, setting a threshold. If node and Spatial adjacency and Then an edge is created with a weight of After adding self-loops and performing symmetric normalization, the comprehensive adjacency matrix is ​​obtained. The graph structure is static and does not involve the construction of intermodal edges.

[0119] Step S3: Extraction of global static embedding and sliding window-based local temporal embedding, specifically,

[0120] Step S3.1: For each spatial node Assign a learnable global static embedding vector This represents the inherent, time-invariant attributes of the geographical location, such as hydrology, substrate composition, and historical susceptibility to algal blooms. Since this embodiment only processes a single image, the global static embedding retains its physical meaning—the prior geographical information of each pixel location. Global Static Embedding Matrix Optimization is performed through backpropagation during network training.

[0121] Step S3.2: Since this embodiment only processes a single image ( The local temporal embedding degenerates into a spectral feature map of the current moment. For modalities... 1D-CNN is used to extract node features, and the kernel size is... Output dimension Since the input sequence length is 1, the convolution operation maintains the output length by padding. ,in The weight matrix to be trained. This represents the bias vector to be trained. The temporal embedding matrix is ​​obtained by combining all nodes. And it does not require cross-modal averaging.

[0122] Step S4: After concatenating the global static embedding and the local temporal embedding, the neighbor information is aggregated on the graph structure using an attention mechanism to update the node temporal embedding. Since this embodiment does not involve temporal prediction, there is no need to perform dynamic relation matrix fusion; the updated temporal embedding is directly used as the fusion feature.

[0123] Step S5: Analyze the fused features through a fully connected layer and the Softmax function, output the probability distribution of algal blooms at each node at the current time, and generate an algal bloom recognition map after binarization.

[0124] Example 3: For the scenario in Example 1, this example provides a multimodal algal bloom recognition and prediction method based on graph attention aggregation to predict algal blooms at different scales and multiple time points in the monitored water area.

[0125] This method is essentially the same as that described in Example 1, and will not be repeated here. The difference lies in the division of the regular spatial units, which involves multiple scales; the smaller the regular spatial unit, the higher the resolution, and vice versa; this embodiment also employs multiple prediction step sizes. At each scale and prediction step size, an independent network model is established, and the steps of this invention are executed. The network model is trained using historical datasets to obtain independent network parameters, thereby enabling the prediction of algal blooms at different scales and prediction times.

[0126] Example 4: For the scenario in Example 1, Figure 5 A schematic diagram of an electronic device 90 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.

[0127] Electronic devices can also refer to various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0128] like Figure 5As shown, the electronic device 90 includes at least one processor 91 and a memory, such as a read-only memory (ROM) 92 or a random access memory (RAM) 93, communicatively connected to the at least one processor 91. The memory stores computer programs executable by the at least one processor. The processor 91 can perform various appropriate actions and processes based on the computer program stored in the ROM 92 or loaded into the RAM 93 from the storage unit 98. The RAM 93 can also store various programs and data required for the operation of the electronic device 90. The processor 91, ROM 92, and RAM 93 are interconnected via a bus 94. An input / output (I / O) interface 95 is also connected to the bus 94.

[0129] Multiple components in electronic device 90 are connected to I / O interface 95, including: input unit 96, such as keyboard, mouse, etc.; output unit 97, such as various types of displays, speakers, etc.; storage unit 98, such as disk, optical disk, etc.; and communication unit 99, such as network card, modem, wireless transceiver, etc. Communication unit 99 allows electronic device 90 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0130] Processor 91 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 91 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 91 performs the various methods and processes described above, such as a multimodal algal bloom recognition and prediction method based on graph attention aggregation.

[0131] In some embodiments, the graph attention aggregation-based multimodal algal bloom recognition and prediction method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 98. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 90 via ROM 92 and / or communication unit 99. When the computer program is loaded into RAM 93 and executed by processor 91, one or more steps of the graph attention aggregation-based multimodal algal bloom recognition and prediction method described above can be performed. Alternatively, in other embodiments, processor 91 can be configured to perform the graph attention aggregation-based multimodal algal bloom recognition and prediction method by any other suitable means (e.g., by means of firmware).

[0132] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0133] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0134] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0135] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0136] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0137] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0138] Finally, it should be noted that the above preferred embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail through the above preferred embodiments, those skilled in the art should understand that various changes can be made to it in form and detail without departing from the scope defined by the claims of the present invention.

Claims

1. A multi-modal algal bloom phenomenon recognition and prediction method based on graph attention aggregation, characterized in that, Includes the following steps: S1: Collect multimodal data of the monitoring area within a continuous time window, and perform spatial alignment, time series construction, and sliding window division on the collected multimodal data; S2: Within each sliding window, a graph structure is independently constructed based on spatial adjacency and cross-modal similarity thresholds to obtain a dynamic cross-modal graph; S3: Extract the global static embedding and the local temporal embedding based on the sliding window for each node in the dynamic cross-modal graph; S4: After concatenating the global static embedding and the local temporal embedding, the attention mechanism is used to aggregate neighbor information on the graph structure, update the node temporal embedding, and calculate the dynamic relationship matrix; S5: Use a one-dimensional temporal convolutional network (TCN) to extract global temporal features from multimodal data and fuse them with the dynamic relation matrix to obtain fused features; S6: The fused features are analyzed through a fully connected layer and the Softmax function to output the probability distribution of algal bloom at each node at the predicted time. After binarization, an algal bloom warning map is generated. The graph structure is specifically: dividing the monitoring area into regular space units, regarding each regular space unit as a node, representing a multi-modal feature vector of a geographical position point; connecting two nodes by a non-directed line as an edge, representing that there is a spatial adjacency or a feature similarity relationship between the two space units, and the edge weight reflects the similarity degree; and the multi-modal data include one or more of multi-source remote sensing images, water surface images, meteorological grid data, and sensor data.

2. The multimodal algal bloom recognition and prediction method based on graph attention aggregation according to claim 1, characterized in that, Step S1 specifically involves: S101: Collect multimodal data of the monitoring area within a continuous time window; S102: Divide the monitoring area into several regular spatial units, and perform spatial interpolation and alignment on all modal data; S103: Let the time window length be... For any time Constructing time series ,in For a moment A multimodal dataset, The total number of modes, For modality At any moment eigenmatrix For modality Feature dimensions; S104: Divide the time series into multiple sliding sub-windows of equal length by rounding down according to the time length.

3. The multimodal algal bloom recognition and prediction method based on graph attention aggregation according to claim 1, characterized in that, Step S2 specifically involves: S201: For any sliding sub-window and modality, calculate the time average of the features of each node within the window; S202: For each modality, calculate the feature cosine similarity between any two nodes. If the two nodes are spatially adjacent and the similarity is greater than the intramodal threshold, then establish an intramodal edge. S203: For any two different modalities, calculate the cross-modal similarity between the two nodes. If the similarity is greater than the cross-modal threshold, then establish an inter-modal edge. S204: Based on the union of modal inner edges and intermodal edges, construct a dynamic cross-modal graph of sliding sub-windows and construct a symmetric normalized comprehensive adjacency matrix.

4. The multimodal algal bloom recognition and prediction method based on graph attention aggregation according to claim 1, characterized in that, Step S3 specifically includes: S301: Assign a learnable global static embedding vector to each node; S302: For each sliding sub-window and each modality, use a one-dimensional standard temporal convolutional network to extract the temporal dynamic features of the nodes; S303: Average and fuse the temporal dynamic features of all modalities to obtain the final temporal embedding of the node, and then combine them to obtain the local temporal embedding matrix.

5. The multimodal algal bloom recognition and prediction method based on graph attention aggregation according to claim 1, characterized in that, Step S4 specifically includes: S401: Concatenate the global static embedding and the local temporal embedding to obtain a fused dual embedding vector; S402: Calculate the unnormalized attention scores of a node to its neighbors, and normalize them using the Softmax function to obtain the attention weights; S403: Use attention weights to perform weighted aggregation of the local temporal embeddings of neighboring nodes to obtain the updated temporal embeddings; S404: Based on the updated temporal embedding, calculate the cosine similarity matrix between nodes as a local dynamic relationship matrix; S405: Use multiple independent attention heads, each head independently calculates calculation steps S402~S404, and concatenate the results of the multiple heads along the feature dimension. The local dynamic relationship matrix is ​​the average value of each head.

6. The multimodal algal bloom recognition and prediction method based on graph attention aggregation according to claim 1, characterized in that, Step S5 specifically includes: S501: For multimodal time series within a time window, a one-dimensional temporal convolutional network is used to extract global temporal features; S502: Divide the global temporal features into multiple segments using a sliding window, multiply the matrix of each segment with the local dynamic relationship matrix, and concatenate the fused features of all windows along the feature dimension to form a fused feature.

7. The method for multimodal algal bloom recognition and prediction based on graph attention aggregation according to claim 1, characterized in that, Step S6 specifically includes: S601: Employs a node-level fully connected layer to output the probability that each node belongs to an algal bloom; S602: After threshold binarization, the spatial distribution prediction map of algal blooms is obtained; S603: Integrate the prediction results with the geographic information system to generate an algal bloom early warning map and mark high-risk areas.

8. The method for multimodal algal bloom recognition and prediction based on graph attention aggregation according to claim 1, characterized in that, The size of the regular spatial unit is selected reasonably according to the needs of the scenario, which depends on the computational efficiency and resolution; the smaller the regular spatial unit, the worse the computational efficiency and the higher the resolution, and vice versa.

9. An electronic device, characterized in that, The method includes at least one processor and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the multimodal algal bloom recognition and prediction method based on graph attention aggregation as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the multimodal algal bloom recognition and prediction method based on graph attention aggregation as described in any one of claims 1 to 8.

Images