A Method and System for Detecting Anomaly Changes in Marine Biological Communities Based on Multimodal Features

Abstract

This invention relates to the field of marine multimodal data processing technology, specifically to a method and system for detecting abnormal changes in marine biological communities based on multimodal features. To address the low accuracy of existing biological community anomaly detection technologies, the method first acquires graph modal data, acoustic modal data, and biochemical modal data from observation points, extracts modal features from each, and then fuses them to obtain an ecological embedding representation. Next, each observation point is treated as a node and connected to form a marine spatial graph structure. Graph convolution processing is used to enhance spatial features, and temporal extraction is combined to obtain the spatiotemporal features of the nodes. Finally, based on the spatiotemporal features of the nodes, outliers and their levels are obtained. A heatmap is then constructed and fused with geographic mapping to form a regional anomaly heatmap. The anomaly levels of the observation points are then marked to obtain the biological community anomaly detection results. This invention, when applied to the field of marine biological community anomaly detection, can improve detection accuracy and efficiency.

Description

Technical Field

[0001] This invention relates to the field of marine multimodal data processing technology, specifically to a method and system for detecting abnormal changes in marine biological communities based on multimodal features. Background Technology

[0002] With increasingly significant changes in the global marine ecological environment, the abnormal evolution of marine biological community structure has become a key indicator for assessing marine health, identifying local ecological crises, and monitoring eutrophication and biodiversity degradation. Accurately capturing abnormal changes in marine biological community structure and achieving early warning and identification is one of the core tasks of intelligent marine monitoring systems.

[0003] While existing technologies have explored the application of deep learning to identify target species in marine images or to track the changing trends of a particular species using time-series analysis, these methods largely focus on single modalities, local features, or short-term changes, making it difficult to construct a comprehensive spatiotemporal dynamic panorama at the community level. Furthermore, anomalous changes in communities often manifest as declining species diversity, loss of functional redundancy, or altered spatial dispersal pathways; single-modal, single-moment analysis strategies struggle to reveal their underlying driving mechanisms. In addition, current anomaly detection models often employ single-threshold methods or static clustering, resulting in slow responses to subtle structural shifts and sudden substitution phenomena in complex communities, frequently leading to high false positive rates and low accuracy. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for detecting abnormal changes in marine biological communities based on multimodal features.

[0005] The technical solution of this invention is as follows:

[0006] A method for detecting anomalous changes in marine biological communities based on multimodal features includes the following operations:

[0007] S1. Obtain graph modal data, acoustic modal data, and biochemical modal data for each observation point in the sea area to be detected. After modal feature extraction, obtain their respective graph modal features, acoustic modal features, and biochemical modal features. Based on the graph modal features, acoustic modal features, and biochemical modal features of each observation point, construct their respective modal feature matrices, obtain their respective modal inner product similarities, and obtain their respective modal covariance maps. After fusing the modal feature matrices and modal covariance maps of each observation point, perform flattening and dimensionality reduction mapping to obtain their respective ecological embedding representations.

[0008] S2. In the sea area to be detected, each observation point is taken as a node, and the nodes are connected by edges to obtain the sea area spatial graph structure; the sea area spatial graph structure is subjected to graph convolution processing to obtain the spatial feature enhancement graph structure; in the spatial feature enhancement graph structure, the spatial structure features of each node are processed by temporal feature extraction to obtain the spatiotemporal features of the respective node sea area.

[0009] S3. Based on the spatiotemporal characteristics of the node sea area, obtain the node anomaly values; based on the mapping relationship between the node anomaly values ​​and the anomaly level standard, obtain the node anomaly level; based on all node anomaly values, construct a heat map and merge it with the geographic mapping to form a regional anomaly heat map; mark the node anomaly level on the regional anomaly heat map to obtain the biological community anomaly detection results.

[0010] In S2, temporal feature extraction is implemented through a gated loop unit. i The operation of processing the spatial structural features of a node by the gated loop unit is implemented through the following formula:

[0011] ,

[0012] ,

[0013] ,

[0014] ,

[0015] For nodes i exist t Constantly update gate features, The Sigmoid activation function is used. To update the gate weight matrix, For nodes i exist t Input characteristics at time step To update the hidden state weight matrix of the gate, For nodes i exist t The spatiotemporal characteristics of the sea area at time -1 To update the gate bias term, For nodes i exist t The reset gate feature at any moment, These are the reset gate weight matrix, the reset gate hidden state weight matrix, and the reset gate bias term, respectively. For nodes i exist t Candidate hidden features at time step These are the candidate weight matrix, the candidate hidden state weight matrix, and the candidate bias term, respectively. For nodes i exist t The spatiotemporal characteristics of the sea area at any given moment.

[0016] In the process of temporal feature extraction in S2, the following steps are also taken: the spatiotemporal features of the sea area at different times of each node are decomposed into multi-level discrete wavelet decomposition to obtain the multi-level spatiotemporal feature components at different times of each node; based on the multi-level spatiotemporal feature components at different times of each node, the multi-level anomaly response intensity and the maximum fluctuation intensity are obtained; nodes with the maximum fluctuation intensity greater than the fluctuation intensity threshold are marked as key anomaly nodes in the biological community anomaly detection results in S3.

[0017] Furthermore, the spatiotemporal features of the key anomaly nodes, the multi-layer anomaly response intensity, and the ecological embedding representation are spliced ​​together to obtain joint diagnostic features. Based on the joint diagnostic features, the probability distribution of the corresponding key anomaly nodes belonging to each anomaly level is obtained, and the anomaly level corresponding to the maximum value of the probability distribution is used as the updated node anomaly level and marked in the biological community anomaly detection results.

[0018] Furthermore, based on the spatiotemporal characteristics of the sea area of ​​key anomaly nodes, the ecological embedding representation, and the anomaly level corresponding to the maximum probability distribution of the key anomaly nodes, a factor diagnosis vector is obtained; based on gradient backpropagation, the factor diagnosis vector is decomposed into influence degree to obtain a multidimensional influence degree value, which is then marked in the anomaly detection results of the biological community.

[0019] In S1, the modal covariance map of the observation points is obtained by multiplying the modal feature matrix of the observation points with the transpose of the modal feature matrix to obtain the orientation similarity matrix; the orientation similarity matrix is ​​then normalized to obtain the modal covariance map of the observation points.

[0020] In S1, the biochemical modal data include seawater temperature, salinity, density, algal bloom concentration, chlorophyll, oxygen demand, and heavy metal ion concentration at the observation points.

[0021] A system for detecting anomalous changes in marine biological communities based on multimodal features, used to implement the aforementioned method for detecting anomalous changes in marine biological communities based on multimodal features, includes:

[0022] The observation point ecological embedding representation generation module is used to acquire graph modal data, acoustic modal data, and biochemical modal data for each observation point in the sea area to be detected. After modal feature extraction, graph modal features, acoustic modal features, and biochemical modal features are obtained respectively. Based on the graph modal features, acoustic modal features, and biochemical modal features of each observation point, a modal feature matrix is ​​constructed, and the modal inner product similarity is obtained to obtain the modal covariance map. After fusing the modal feature matrix and modal covariance map of each observation point, flattening and dimensionality reduction mapping are performed to obtain the respective ecological embedding representation.

[0023] The node-based sea area spatiotemporal feature generation module is used to treat each observation point in the sea area to be detected as a node, connect the nodes with edges to obtain the sea area spatial graph structure; perform graph convolution processing on the sea area spatial graph structure to obtain the spatial feature enhancement graph structure; in the spatial feature enhancement graph structure, the spatial structure features of each node are processed by temporal feature extraction to obtain their respective node sea area spatiotemporal features;

[0024] The biological community anomaly detection result generation module is used to obtain node anomaly values ​​based on the spatiotemporal characteristics of the node's marine area; obtain the node's anomaly level based on the mapping relationship between the node anomaly values ​​and the anomaly level standard; construct a heat map based on all node anomaly values ​​and fuse it with the geographic mapping to form a regional anomaly heat map; mark the node anomaly level on the regional anomaly heat map to obtain the biological community anomaly detection result.

[0025] A device for detecting abnormal changes in marine biological communities based on multimodal features includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the above-mentioned method for detecting abnormal changes in marine biological communities based on multimodal features.

[0026] A computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described method for detecting abnormal changes in marine biological communities based on multimodal features.

[0027] The beneficial effects of this invention are as follows:

[0028] This invention provides a method for detecting abnormal changes in marine biological communities based on multimodal features. First, it acquires graph modal data, acoustic modal data, and biochemical modal data from observation points, extracts modal features from each, and then fuses them. By integrating the dynamic change features of the three modalities, an ecological embedding representation covering visual, acoustic, and biochemical indicators is formed, thereby more comprehensively and accurately capturing the overall ecological status of the water body at the observation point and significantly improving the understanding and representation of complex aquatic ecosystems. Then, each observation point is treated as a node and connected to form a marine spatial graph structure. Spatial features are enhanced through graph convolution processing, and then combined with temporal extraction to obtain the spatiotemporal features of the node's marine area. This invention effectively captures the spatial dependence characteristics between observation points, clearly reconstructing the spatial diffusion paths and overall evolution trends of pollutants and ecological anomalies. Finally, based on the spatiotemporal characteristics of the node sea area, it obtains anomaly values ​​and anomaly levels, and then constructs a heat map and integrates it with geographic mapping to form a regional anomaly heat map. By marking the anomaly levels of observation points, it obtains the results of biological community anomaly detection. This not only improves the accuracy of biological community anomaly detection at observation points through precise node anomaly analysis, but also realizes intuitive visualization of anomaly information through regional anomaly heat maps, making the detection results easier to understand and apply. When applied to the field of marine biological community anomaly change detection, this invention can improve detection accuracy and efficiency. Attached Figure Description

[0029] The solutions and advantages of this application will become clear to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of the invention.

[0030] In the attached diagram:

[0031] Figure 1 This is a schematic diagram of the method flow in this embodiment;

[0032] Figure 2 This is a comparison chart of the accuracy rates of various methods in the embodiments;

[0033] Figure 3 This is a comparison chart of the false alarm rate and false positive rate of various methods in the embodiments;

[0034] Figure 4 This is a comparison chart showing the accuracy of various methods explained in the examples;

[0035] Figure 5 This is a comparison chart of the inference times of various methods in the embodiments;

[0036] Figure 6 The above is a radar chart comparing the key performance of various methods in the embodiments. Detailed Implementation

[0037] Exemplary embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings.

[0038] This embodiment provides a method for detecting anomalous changes in marine biological communities based on multimodal features. (See also...) Figure 1 This includes the following operations:

[0039] S1. Obtain graph modal data, acoustic modal data, and biochemical modal data for each observation point in the sea area to be detected. After modal feature extraction, obtain their respective graph modal features, acoustic modal features, and biochemical modal features. Based on the graph modal features, acoustic modal features, and biochemical modal features of each observation point, construct their respective modal feature matrices, obtain their respective modal inner product similarities, and obtain their respective modal covariance maps. After fusing the modal feature matrices and modal covariance maps of each observation point, perform flattening and dimensionality reduction mapping to obtain their respective ecological embedding representations.

[0040] S2. In the sea area to be detected, each observation point is taken as a node, and the nodes are connected by edges to obtain the sea area spatial graph structure; the sea area spatial graph structure is subjected to graph convolution processing to obtain the spatial feature enhancement graph structure; in the spatial feature enhancement graph structure, the spatial structure features of each node are processed by temporal feature extraction to obtain the spatiotemporal features of the respective node sea area.

[0041] S3. Based on the spatiotemporal characteristics of the node sea area, obtain the node anomaly values; based on the mapping relationship between the node anomaly values ​​and the anomaly level standard, obtain the node anomaly level; based on all node anomaly values, construct a heat map and merge it with the geographic mapping to form a regional anomaly heat map; mark the node anomaly level on the regional anomaly heat map to obtain the biological community anomaly detection results.

[0042] The specific steps and details are as follows.

[0043] S1. Obtain graph modal data, acoustic modal data, and biochemical modal data for each observation point in the sea area to be detected. After modal feature extraction, obtain their respective graph modal features, acoustic modal features, and biochemical modal features. Based on the graph modal features, acoustic modal features, and biochemical modal features of each observation point, construct their respective modal feature matrices, obtain their respective modal inner product similarity, and obtain their respective modal covariance maps. After fusing the modal feature matrix and modal covariance map of each observation point, perform flattening and dimensionality reduction mapping to obtain their respective ecological embedding representations.

[0044] The system acquires graphical, acoustic, and biochemical modal data from observation points and extracts modal features from each. This successfully achieves semantic alignment and structural difference elimination of heterogeneous multi-source data from images, acoustics, and biochemistry, allowing data such as water color, underwater acoustic signatures, and nutrient factors, which originally had different forms and meanings, to be effectively correlated within a unified framework. Subsequently, the extracted multimodal features are fused. By integrating the dynamic change features of the three modalities, an ecological embedding representation covering visual, acoustic, and biochemical indicators is formed. This allows for a more comprehensive and accurate capture of the overall ecological status of the water bodies at the observation points, significantly improving the understanding and representation capabilities of complex aquatic ecosystems.

[0045] First, acquire graphical modal data, acoustic modal data, and biochemical modal data for each observation point (or the detection area where the observation point is located) in the sea area to be detected.

[0046] The graph modal data is continuously acquired through platforms such as drone aerial photography, underwater cameras, or buoy monitoring. It visually reflects the morphology of biological communities, population density, spatial aggregation, and anomalous shapes. The acoustic modal data consists of acoustic echo maps of the monitoring area where the observation point is located. This data can be acquired using equipment such as multibeam sonar or side-scan sonar. It reflects behavioral disturbance characteristics such as changes in target density and behavioral reflection intensity at the observation point. The biochemical modal data includes data on seawater temperature, salinity, density, algal bloom concentration, chlorophyll, oxygen demand, and heavy metal ion concentration at the observation point. This data is acquired through fixed buoys or real-time water sampling equipment and reflects environmental characteristics such as ecological stress.

[0047] Then, modal features were extracted from the graph modal data, acoustic modal data, and biochemical modal data of each observation point to obtain their respective graph modal features, acoustic modal features, and biochemical modal features.

[0048] Modal feature extraction from graph modal data. Graph modal data (images) from observation points are easily affected by changes in shooting angle, water transparency, and lighting conditions, leading to unstable feature representation capabilities. Furthermore, seawater images contain a large amount of redundant background information, such as water texture, floating objects, and benthic sediments, which can also interfere with target community feature extraction. This embodiment aims to improve the expressive power of graph modal data from observation points by performing modal feature extraction processing on the graph modal data to obtain graph modal features.

[0049] Modal feature extraction from graph modal data can be achieved by using layer-by-layer convolution, non-linear activation, and pooling operations to extract high-order semantic features from the original image. This can be calculated using the following formula:

[0050] ,

[0051] in, Graph modal features are static representations of community states within an image modality, facilitating the capture of morphological features such as density, texture, and color distribution. For graph modal data, Indicates the first i The convolutional module includes a Conv-BN-ReLU structure. This indicates that global average pooling is performed on each channel of the image to obtain the average response of each channel.

[0052] Modal feature extraction from acoustic modal data can be achieved by performing two-dimensional convolution on the acoustic modal data (acoustic echo map) to identify local spectral structure (multi-peak echoes) and temporal energy information. The specific calculation formula is as follows:

[0053] ,

[0054] Acoustic modal characteristics, reflecting the intensity of ecological disturbance and community activity. S For acoustic modal data, It is a 3×3 two-dimensional convolution.

[0055] Modal feature extraction from biochemical modal data can be achieved by projecting the biochemical modal data into a feature space consistent with that of graph modal data or acoustic modal data using a linear mapping layer. The specific calculation formula is as follows:

[0056] ,

[0057] Biochemical modal characteristics encode information about how environmental factors affect community state, providing a priori signals as to whether a community is in an unsuitable environment. C For biochemical modal data, Extracting weight matrices for biochemical features. This is the biochemical bias vector; These are biochemical modal characteristics.

[0058] In some jellyfish outbreaks, the image channel can detect dense, milky-white clumps, the acoustic channel reflects high-frequency energy accumulation, and the biochemical channel shows decreased dissolved oxygen and changes in salinity. Modeling this information in isolation would miss the interconnected nature of these community anomalies. Traditional modal fusion mechanisms, such as simple stitching, modal weight gating, or Transformer weighting, cannot characterize the ecological structural characteristics of such intermodal cooperative perturbations.

[0059] Therefore, this embodiment then constructs a modal feature matrix based on the graph modal features, acoustic modal features, and biochemical modal features of each observation point. The modal feature matrix is ​​formed by splicing graph modal features, acoustic modal features, and biochemical modal features.

[0060] Subsequently, based on the modal feature matrix of each observation point, the modal inner product similarity is obtained, resulting in the modal covariance map of each observation point. Specifically, the modal feature matrix of the observation point is multiplied by its transpose to obtain the orientation similarity matrix; the orientation similarity matrix is ​​then normalized to obtain the modal covariance map of the observation point.

[0061] Normalization can be achieved using the following formula:

[0062] ,

[0063] B This is a modal covariance map, which describes the dynamic dependencies between modes. g This is a direction similarity matrix, where each element represents the feature direction similarity between different modalities. The scaling factor controls the inner product magnitude. The softmax operation normalizes each row in the modality dimension, ensuring that the sum of the importance weights of each modality to the other modalities is 1. This operation does not rely on any labels; it learns a structured association graph by observing changes in feature responses between modalities, automatically discovering which modalities are more suitable for describing a certain type of anomaly, and automatically adjusting the perceptual weights during the inference phase.

[0064] Finally, the modal feature matrix and modal covariance map of each observation point are fused (through multiplication) to reweight the features of each modality, thereby strengthening the similarity enhancement between modalities (for example, during abnormal algal expansion, the green response in the image and the chlorophyll concentration in the chemical modality are synergistically enhanced, increasing their weight and incorporating more biochemical response information into the image representation). Flattening and dimensionality reduction mapping are then performed to obtain their respective ecological embedding representations. The ecological embedding representation visualizes the time-slice ecological vector after multimodal coupling enhancement. Internally, it encodes the dynamic weight changes of image boundary variations, acoustic disturbance responses, and environmental factor fluctuations during ecological evolution, serving as input features for subsequent community evolution trends and completing the evolutionary trajectory in time and space.

[0065] The dimensionality reduction mapping operation described above can be implemented using a multilayer perceptron, which contains two fully connected layers and an activation dimensionality reduction mapping function.

[0066] S2. In the sea area to be detected, each observation point is taken as a node, and the nodes are connected by edges to obtain the sea area spatial graph structure; the sea area spatial graph structure is subjected to graph convolution processing to obtain the spatial feature enhancement graph structure; in the spatial feature enhancement graph structure, the spatial structure features of each node are processed by temporal feature extraction to obtain the spatiotemporal features of the respective node sea area.

[0067] By treating each observation point as a node and connecting them to form a marine spatial map structure, spatial features are enhanced through graph convolution processing. Combined with temporal extraction, the spatiotemporal features of the marine area at each node are obtained, which can effectively capture the spatial dependence features between observation points and clearly restore the spatial diffusion paths and overall evolution trends of pollutants and ecological anomalies. At the same time, through spatiotemporal feature characterization, the dynamic trajectory of anomaly propagation between communities can be accurately tracked, providing more comprehensive spatial correlation information for anomaly detection, thereby significantly improving the accuracy of biological community anomaly detection.

[0068] The process of anomalous evolution in marine biological communities does not occur in isolated locations, but rather spreads gradually throughout the ecological continuum, exhibiting significant neighborhood responsiveness and regional propagation. For example, red tides often spread gradually outward from eutrophic centers, controlled by marine geographical factors such as ocean current structure, temperature gradients, and wind disturbances. This evolutionary pattern manifests in data as follows: an anomalous signal is first detected at an observation point, followed by secondary response signals in neighboring areas.

[0069] First, to facilitate the demonstration of this spatially dependent structure, this embodiment treats each observation point in the sea area to be detected as a node, and connects the nodes with edges to obtain the sea area spatial map structure.

[0070] In the spatial map structure of the sea area, the edge weights between nodes are:

[0071] ,

[0072] For nodes i ( ) and nodes j ( The edge weights of ) For nodes i and nodes j The distance is the Euclidean geographical distance obtained after mapping latitude and longitude. For spatial scale normalization factor, For nodes i and nodes j The degree of structural topological coupling is obtained based on the ocean circulation, ecological functional zones, and pollution sources of nodes i and j. For example, if and If they belong to the same ocean circulation, ecological functional zone, or are affected by the same pollution source, they are assigned a higher coupling value (such as 1); otherwise, they are assigned 0 or a small weight.

[0073] All edge weights constitute the initial adjacency matrix. After symmetric normalization, the normalized adjacency matrix, which serves as the adjacency matrix for the marine spatial map structure, can be obtained using the following formula:

[0074] ,

[0075] ,

[0076] The adjacency matrix of the marine spatial map structure represents the spatial ecological relationships among all observation points. D For degree matrix, A This is the initial adjacency matrix.

[0077] Furthermore, in ecological anomaly detection scenarios, the state of marine biological communities is significantly affected by changes in the state of neighboring areas. Local eutrophication of water bodies can lead to rapid algal proliferation in adjacent areas; acoustic disturbance reflection anomalies can spatially extend to upstream and downstream water bodies. If the state transfer mechanism in the spatial graph structure is ignored and judgment is made solely based on local node information, key diffusion paths and anomaly linkage patterns will be missed, reducing the breadth of detection and response prediction capabilities.

[0078] Therefore, in this embodiment, the marine spatial graph structure is subjected to graph convolution processing, and the node states in the marine spatial graph structure are fused with the states of their adjacent nodes to obtain a new regional representation, thereby achieving structural propagation and obtaining a spatial feature-enhanced graph structure.

[0079] The graph convolution operation can be implemented using the following formula:

[0080] ,

[0081] Z This is a spatial feature enhancement graph structure that incorporates the influence of neighborhood features. The adjacency matrix of the marine spatial map structure. G This is the ecological embedding feature matrix formed by the ecological embedding representation of all nodes corresponding to the observation points in the spatial feature enhancement graph structure. The graph convolution weight matrix, It is the ReLU activation function, used to enhance nonlinear modeling capabilities.

[0082] Finally, in the spatial feature enhancement graph structure, the spatial structure features of each node are processed by temporal feature extraction to capture the temporal evolution trajectory of the community and form a hidden state description, thus obtaining the spatiotemporal features of the respective node's sea area.

[0083] The aforementioned temporal feature extraction process is implemented through a gated loop unit, where the node... i The spatial structural features processed by the gated loop unit are implemented through the following formula:

[0084] ,

[0085] ,

[0086] ,

[0087] ,

[0088] For nodes i exist t Constantly update gate features, The Sigmoid activation function is used. To update the gate weight matrix, For nodes i exist t Input features at any given time (ecological embedding representation of nodes). To update the hidden state weight matrix of the gate, For nodes i exist t The spatiotemporal characteristics of the sea area at time -1 To update the gate bias term, For nodes i exist t The reset gate feature at any moment, These are the reset gate weight matrix, the reset gate hidden state weight matrix, and the reset gate bias term, respectively. For nodes i exist t Candidate hidden features at time step These are the candidate weight matrix, the candidate hidden state weight matrix, and the candidate bias term, respectively. For nodes i exist t The spatiotemporal characteristics of the sea area at any given moment.

[0089] S3. Based on the spatiotemporal characteristics of the node sea area, obtain the node anomaly values ​​and node anomaly levels; based on all node anomaly values, construct a heat map and fuse it with the geographic mapping to form a regional anomaly heat map; mark the node anomaly levels on the regional anomaly heat map to obtain the biological community anomaly detection results.

[0090] Anomalies and anomaly levels are obtained based on the spatiotemporal characteristics of the sea area at the nodes. Then, a regional anomaly heatmap is formed by constructing a heatmap and integrating it with geographic mapping. The anomaly level of the observation points is marked to obtain the anomaly detection results of the biological community. This not only improves the accuracy of the anomaly detection of the biological community at the observation points by using precise node anomaly analysis, but also realizes the intuitive visualization of the anomaly information through the regional anomaly heatmap, making the detection results easier to understand and apply.

[0091] First, based on the spatiotemporal characteristics of the sea area at each node, outlier values ​​can be obtained using the following formula:

[0092] ,

[0093] For nodes i Outliers reflect node i The biological anomalies in the area corresponding to the observation point For nodes i exist T The spatiotemporal characteristics of the sea area at the final moment. represents the spatiotemporal average characteristics of the nodal sea area, and || ||2 represents the L2 norm.

[0094] Then, based on the mapping relationship between node outliers and outlier level standards, the outlier level of a node is obtained.

[0095] Finally, a heatmap is constructed based on the outliers of all nodes and fused with the geographic mapping to form a regional anomaly heatmap. The anomaly level of each node is marked on the regional anomaly heatmap to obtain the anomaly detection results of the biological community.

[0096] In the above scheme, dynamic anomaly detection of biological communities can be achieved by acquiring continuous time-series anomaly detection results and displaying the change information in a heat map.

[0097] Furthermore, in identifying anomalous changes in ecological communities, the frequency and patterns of state fluctuations vary significantly across different regions. For example, sea surface temperature changes slowly over a diurnal cycle, while phytoplankton concentration may fluctuate dramatically in the short term due to sudden surges in eutrophication. Especially in community evolution, anomalous fluctuations often exhibit multi-stage, multi-frequency characteristics, such as low-amplitude initiation, high-frequency response, and strong disturbance diffusion. Traditional modeling methods at a uniform scale struggle to simultaneously capture early subtle disturbances and mid-to-late-stage strong changes, potentially missing precursor signals or misidentifying structural noise. To improve the accuracy of anomaly detection results in this embodiment, the accuracy of the anomaly detection results has been further optimized, highlighting the areas where key observation points are located and adding explanations of anomaly information. This enhances the readability and ecological controllability of the detection results, facilitating efficient subsequent control and management. Specific details are as follows.

[0098] Key anomaly nodes (key observation points) are identified. The temporal feature extraction process in S2 includes performing multi-level discrete wavelet decomposition on the marine spatiotemporal features of each node at different times to obtain its respective multi-level spatiotemporal feature components at different times. Based on these multi-level spatiotemporal feature components, the multi-level anomaly response intensity and maximum fluctuation intensity of each node are obtained. Nodes with a maximum fluctuation intensity greater than the fluctuation intensity threshold are designated as key anomaly nodes and marked in the biological community anomaly detection results in S3. The areas where the observation points corresponding to key anomaly nodes are located exhibit strong anomaly behavior under multi-scale responses, possessing characteristics of strong spatial localization, high fluctuation sensitivity, and concentrated discriminative information.

[0099] Among them, nodes i exist l The intensity of the abnormal response of a layer can be obtained using the following formula:

[0100] ,

[0101] For nodes i exist l The abnormal response intensity of the layer, For nodes i exist l Layer t High-frequency components at time (by nodes) i exist t The spatiotemporal characteristics of the sea area at a given time (obtained through discrete wavelet decomposition) reflect the nodes. i In the l Fluctuation patterns at various scales, when l≥ At time 1, discrete wavelet decomposition performs high-frequency decomposition. l= At time 0, discrete wavelet decomposition performs low-frequency decomposition. T The time is the final moment. For nodes i exist l Layer feature sequence length.

[0102] node i The maximum fluctuation intensity is the maximum value of the multi-level anomaly response intensity after frequency band normalization. Frequency band normalization is achieved using the following formula:

[0103] ,

[0104] For nodes i exist l The normalized anomaly response intensity of the layer is the average anomaly response intensity of all nodes. The standard deviation of the abnormal response intensity of all nodes. It is a stabilizing factor.

[0105] Furthermore, to optimize the inspection results of key anomaly nodes, this embodiment concatenates the spatiotemporal characteristics of the node's marine area, multi-layer anomaly response intensity, and ecological embedding representation to obtain joint diagnostic features. These joint diagnostic features contain the node's overall ecological change trend, response intensity at different frequency bands, and environmental modal information such as color, hydrology, and acoustic signatures of its location. Based on these joint diagnostic features, the probability distribution of each anomaly level for the corresponding key anomaly node is obtained. The anomaly level corresponding to the maximum probability distribution value is used as the updated node anomaly level and marked in the biological community anomaly detection results.

[0106] Among them, key abnormal nodes v The anomaly level is calculated using the following formula:

[0107] ,

[0108] ,

[0109] in, y v Key abnormal nodes v The probability distribution vector belonging to each anomaly level. It is a two-layer perceptron network. d v For joint diagnostic features, the softmax operation ensures that the sum of probabilities is 1, and argmax represents the index of the maximum value. Indicates finding that makes The largest category index k, Key abnormal nodes v The abnormal level.

[0110] Furthermore, to provide an ecological interpretation of anomalies in biological communities at observation points, this embodiment obtains a factor diagnosis vector based on the spatiotemporal characteristics of the nodal marine area of ​​key anomaly nodes, ecological embedding representation, and anomaly level corresponding to the maximum probability distribution. Based on gradient backpropagation, the factor diagnosis vector is decomposed into influence degree to obtain multidimensional influence degree values, which are then marked in the anomaly detection results of biological communities. This provides interpretable factor weight decomposition, natural language structured diagnosis, and spatial visualization output maps for the anomaly detection results of biological communities, meeting the core needs of multiple scenarios such as marine management, scientific research, and emergency response.

[0111] The aforementioned factor diagnostic vector is a feature vector formed by splicing together the spatiotemporal characteristics of the node's sea area, the ecological embedding representation, and the anomaly level corresponding to the maximum probability distribution.

[0112] The operation to obtain the multidimensional influence value can be achieved using the following formula:

[0113] ,

[0114] Key abnormal nodes v The multidimensional influence value is a vector. , Key abnormal nodes v The modal factor influence value reflects the importance of modal factors (color / acoustics / texture). Key abnormal nodes v The spatiotemporal state influence value reflects the importance of spatiotemporal characteristics. Key abnormal nodes v The multi-level anomaly response influence degree value reflects the importance of the multi-level anomaly response; Key abnormal nodes v The factor diagnostic vector of the first n Features in each dimension It can be an ecological embedding representation, or a spatiotemporal feature of a node's sea area, or an anomaly level corresponding to the maximum value of a probability distribution.

[0115] To verify the effectiveness of the method in this embodiment, the following experiment was conducted.

[0116] To systematically verify the performance advantages of the method in this embodiment (corresponding to the method of the present invention in the accompanying drawings) in the detection of marine biological community anomalies, a multimodal ecological observation dataset based on typical nearshore stations in the East China Sea was constructed. The dataset consists of three modalities: ① Image modality: including images captured by UAV cruise and Sentinel-2 remote sensing images, reflecting water surface color, algal density, and spatial distribution; ② Acoustic modality: underwater acoustic recordings from buoys to identify zooplankton activity frequency, population disturbance, and changes in background noise; ③ Biochemical modality: indicators such as seawater temperature, salinity, density, algal bloom concentration, chlorophyll, oxygen demand, heavy metal ion concentration, and pH, used to model the evolution of internal ecological parameters in the water body. The dataset is labeled with four ecological states: ① Normal water body; ② Algal bloom outbreak (e.g., concentrated areas of red tide masses); ③ Nutrient imbalance (hypoxia, abnormal pH, etc.); ④ Water disturbance (sudden animal aggregation or migration). A total of 530 sets of multimodal observation samples were constructed, of which 380 sets were used for training and 150 sets were used for testing, ensuring coverage of different sea states, lighting conditions, and wind speeds.

[0117] To test the robustness and multimodal feature representation capability of the method in this embodiment under complex real-world scenarios, the following four mainstream existing methods were selected for comparison: ① ResNet-18: a typical unimodal image classifier that extracts image texture and color change features; ② LSTM: a unimodal trend detection method based on time series modeling that uses only biochemical indicators; ③ Concat-MLP: a fusion method that directly concatenates three-modal vectors into an MLP for classification; ④ GraphSAGE: a graph neural network that introduces regional graph structure relationships to learn the ecological spatial structure.

[0118] All methods were evaluated under the same training and test set conditions. The test perturbation conditions included: no perturbation (0.0): ideal conditions; moderate perturbation (0.5): random illumination shift, noise injection; and high perturbation (1.0): mixed occlusion, mode loss, and scale variation. Evaluation metrics: ① Accuracy (ACC): the overall correctness of anomaly and normal classification; ② False negative rate (MR): the proportion of anomaly samples that are predicted as normal; ③ False positive rate (FAR): the proportion of normal samples that are misclassified as anomalies; ④ Interpretive accuracy (Interp-Acc): the proportion of factor interpretations output by the system that are consistent with expert annotations; ⑤ Inference time (Time): the time required to process a single set of input data, measuring the system deployment efficiency. The comparative experimental results are shown in Table 1.

[0119] Table 1. Comparison of data from different methods under five major indicators.

[0120]

[0121] Because ResNet-18 and LSTM cannot provide structured causal explanations, they are not evaluable (N / A) in the Interp-Acc metric. ResNet-18 is a unimodal image classification network; its output is merely a "whether it is abnormal" label prediction, unable to correlate ecological factors or the source of the anomaly, and lacks causal modeling capabilities. While LSTM is suitable for temporal prediction, it lacks regional structure modeling and factor significance scoring mechanisms, and cannot determine whether an anomaly is triggered by "nutrient disturbance" or "spatial clustering," etc.

[0122] The five indicators in Table 1 are evaluated using different methods and cannot be directly displayed in the same radar chart. This invention retains the positive indicators (higher values ​​are better, i.e., accuracy and segmentation precision) unchanged. For the negative indicators (lower values ​​are better, i.e., false negative rate, concentration error, and inference time), the experiment uses a "1 - value" inverse processing to uniformly convert all indicators into a form where higher values ​​are better, so that they can be displayed in the same chart. Since the ResNet-18 indicator is the worst among all methods, its normalized value is 0 (or very close to 0). When multiple such indicators appear, it indicates that the method shrinks back to the center point in multiple directions on the radar chart, thus appearing as a straight line or a point. To avoid this problem, a "biased offset ε" mechanism is introduced in the experiment to forcibly compress the normalized value to the range [ε, 1 - ε], ensuring that all methods have at least some visual display space in the chart.

[0123] See the experimental results for the diagram. Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6In complex multimodal ecosystems, existing methods exhibit significant performance limitations. For example, ResNet-18, as an image classification backbone, fails to identify ecologically aberrant targets with blurred boundaries without spatial topology or temporal modeling, resulting in a false negative rate as high as 23.4%. While LSTM is suitable for biochemical indicator sequence analysis, its lack of coupling between modalities leads to a still high false positive rate. Concat-MLP, as a direct splicing multimodal fusion model, improves overall perception capabilities, but due to the lack of structural information modeling, its interpretation accuracy is only 56.3%, making it difficult to support ecological causal judgment tasks. GraphSAGE enhances spatial diffusion perception capabilities through region structure mapping, but its long processing time (1.05s) makes it difficult to meet real-time requirements.

[0124] In comparison, the method in this embodiment achieves a systematic breakthrough in multiple dimensions: the accuracy reaches 91.2%, the highest value; the false negative rate and false positive rate are reduced to 6.2% and 6.1% respectively, which is better than all existing comparative methods; the diagnosis time is only 0.68 seconds, which is optimal, demonstrating excellent reasoning efficiency; the accuracy of ecological factor interpretation is as high as 90.2%, which is significantly better than graph structure methods and splicing models.

[0125] In summary, the method in this embodiment not only has significant advantages in diagnostic accuracy and structural modeling capabilities, but also achieves optimal performance in both real-time performance and interpretability, fully demonstrating its application value in multimodal intelligent anomaly detection for marine biological communities and possessing broad engineering deployment potential and ecological decision support capabilities.

[0126] This embodiment also provides a marine biological community anomaly change detection system based on multimodal features, used to implement the above-mentioned marine biological community anomaly change detection method based on multimodal features, including:

[0127] The observation point ecological embedding representation generation module is used to acquire graph modal data, acoustic modal data, and biochemical modal data for each observation point in the sea area to be detected. After modal feature extraction, graph modal features, acoustic modal features, and biochemical modal features are obtained respectively. Based on the graph modal features, acoustic modal features, and biochemical modal features of each observation point, a modal feature matrix is ​​constructed, and the modal inner product similarity is obtained to obtain the modal covariance map. After fusing the modal feature matrix and modal covariance map of each observation point, flattening and dimensionality reduction mapping are performed to obtain the respective ecological embedding representation.

[0128] The node-based sea area spatiotemporal feature generation module is used to treat each observation point in the sea area to be detected as a node, connect the nodes with edges to obtain the sea area spatial graph structure; perform graph convolution processing on the sea area spatial graph structure to obtain the spatial feature enhancement graph structure; in the spatial feature enhancement graph structure, the spatial structure features of each node are processed by temporal feature extraction to obtain their respective node sea area spatiotemporal features;

[0129] The biological community anomaly detection result generation module is used to obtain node anomaly values ​​based on the spatiotemporal characteristics of the node's marine area; obtain the node's anomaly level based on the mapping relationship between the node anomaly values ​​and the anomaly level standard; construct a heat map based on all node anomaly values ​​and fuse it with the geographic mapping to form a regional anomaly heat map; mark the node anomaly level on the regional anomaly heat map to obtain the biological community anomaly detection result.

[0130] This embodiment also provides a marine biological community anomaly change detection device based on multimodal features, including a processor and a memory, wherein the processor executes the computer program stored in the memory to implement the above-mentioned marine biological community anomaly change detection method based on multimodal features.

[0131] This embodiment also provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the above-described method for detecting abnormal changes in marine biological communities based on multimodal features.

[0132] This embodiment provides a method for detecting abnormal changes in marine biological communities based on multimodal features. First, it acquires graph modal data, acoustic modal data, and biochemical modal data from observation points, extracts modal features from each, and then fuses them. By integrating the dynamic change features of the three modalities, an ecological embedding representation covering visual, acoustic, and biochemical indicators is formed, thereby more comprehensively and accurately capturing the overall ecological status of the water body at the observation points and significantly improving the understanding and representation of complex aquatic ecosystems. Then, each observation point is treated as a node and connected to form a marine spatial graph structure. Spatial features are enhanced through graph convolution processing, and then combined with temporal extraction to obtain the spatiotemporal features of the node marine area. This invention effectively captures the spatial dependence characteristics between observation points, clearly reconstructing the spatial diffusion paths and overall evolution trends of pollutants and ecological anomalies. Finally, based on the spatiotemporal characteristics of the node sea area, it obtains anomaly values ​​and anomaly levels, and then constructs a heat map and integrates it with geographic mapping to form a regional anomaly heat map. By marking the anomaly levels of observation points, it obtains the results of biological community anomaly detection. This not only improves the accuracy of biological community anomaly detection at observation points through precise node anomaly analysis, but also realizes intuitive visualization of anomaly information through regional anomaly heat maps, making the detection results easier to understand and apply. When applied to the field of marine biological community anomaly change detection, this invention can improve detection accuracy and efficiency.

Claims

1. A method for detecting anomalous changes in marine biological communities based on multimodal features, characterized in that, This includes the following operations: S1. Obtain the graph modal data, acoustic modal data and biochemical modal data of each observation point in the sea area to be detected, and obtain their respective graph modal features, acoustic modal features and biochemical modal features after modal feature extraction; Based on the graphical, acoustic, and biochemical modal features of each observation point, a modal feature matrix is ​​constructed for each point. The modal inner product similarity is then obtained to acquire the modal covariance map for each point. The modal covariance map for each observation point is obtained by multiplying the modal feature matrix of the observation point with its transpose to obtain the orientation similarity matrix. The orientation similarity matrix is ​​then normalized to obtain the modal covariance map for the observation point. After fusing the modal feature matrix and modal covariance map of each observation point, flattening and dimensionality reduction mapping are performed to obtain their respective ecological embedding representations; S2. Treat each observation point in the sea area to be detected as a node, connect the nodes with edges to obtain the sea area spatial graph structure; perform graph convolution on the sea area spatial graph structure to obtain the spatial feature enhancement graph structure. In the spatial feature enhancement graph structure, the spatial structural features of each node are processed by temporal feature extraction to obtain the spatiotemporal features of the respective node's sea area. S3. Based on the spatiotemporal characteristics of the sea area at each node, obtain the node outlier values; based on the mapping relationship between the node outlier values ​​and the outlier level standard, obtain the node's outlier level; based on all node outlier values, construct a heat map and integrate it with the geographic mapping to form a regional outlier heat map. Mark the anomaly level of nodes on the regional anomaly heatmap to obtain the anomaly detection results of biological communities; In the process of temporal feature extraction, the process also includes performing multi-level discrete wavelet decomposition on the sea area spatiotemporal features of each node at different times to obtain the multi-level spatiotemporal feature components at different times. Based on the multi-layer spatiotemporal feature components of each node at different times, the multi-layer anomaly response intensity and maximum fluctuation intensity of each node are obtained; i exist l The intensity of the layer's abnormal response is achieved using the following formula: ， For nodes i exist l The abnormal response intensity of the layer, For nodes i exist l Layer t The high-frequency components at time t, reflecting the node i In the l Fluctuation patterns at various scales, when l≥ At time 1, discrete wavelet decomposition performs high-frequency decomposition. l= At time 0, discrete wavelet decomposition performs low-frequency decomposition. T The time is the final moment. For nodes i exist l Layer feature sequence length; node i The maximum fluctuation intensity is the maximum value of the multi-level anomaly response intensity after frequency band normalization. Nodes with maximum fluctuation intensity greater than the fluctuation intensity threshold are designated as key anomaly nodes. The spatiotemporal features of the key anomaly nodes, the multi-layer anomaly response intensity, and the ecological embedding representation are concatenated to obtain joint diagnostic features. Based on the joint diagnostic features, the probability distribution of the corresponding key anomaly node belonging to each anomaly level is obtained. The anomaly level corresponding to the maximum value of the probability distribution is used as the updated node anomaly level and marked in the biological community anomaly detection results.

2. The method for detecting abnormal changes in marine biological communities based on multimodal features according to claim 1, characterized in that, In S2, temporal feature extraction is implemented through gated recurrent units. No. i The operation of processing the spatial structural features of a node by the gated loop unit is implemented through the following formula: ， ， ， ， For nodes i exist t Constantly update gate features, The Sigmoid activation function is used. To update the gate weight matrix, For nodes i exist t Input characteristics at time step To update the hidden state weight matrix of the gate, For nodes i exist t The spatiotemporal characteristics of the sea area at time -1 To update the gate bias term, For nodes i exist t The reset gate feature at any moment, These are the reset gate weight matrix, the reset gate hidden state weight matrix, and the reset gate bias term, respectively. For nodes i exist t Candidate hidden features at time step These are the candidate weight matrix, the candidate hidden state weight matrix, and the candidate bias term, respectively. For nodes i exist t The spatiotemporal characteristics of the sea area at any given moment.

3. The method for detecting abnormal changes in marine biological communities based on multimodal features according to claim 1, characterized in that, Also includes Based on the spatiotemporal characteristics of key anomaly nodes in the sea area, their ecological embedding representation, and the anomaly level corresponding to the maximum probability distribution, a factor diagnosis vector is obtained. Based on gradient backpropagation, the influence degree of the factor diagnosis vector is decomposed to obtain multidimensional influence degree values, which are then marked in the anomaly detection results of biological communities.

4. The method for detecting abnormal changes in marine biological communities based on multimodal features according to claim 1, characterized in that, In S1, the biochemical modal data include seawater temperature, salinity, density, algal bloom concentration, chlorophyll, oxygen demand, and heavy metal ion concentration at the observation points.

5. A system for detecting abnormal changes in marine biological communities based on multimodal features, used to implement the method for detecting abnormal changes in marine biological communities based on multimodal features as described in claim 1, characterized in that, include: The observation point ecological embedding representation generation module is used to acquire the graph modal data, acoustic modal data and biochemical modal data of each observation point in the sea area to be detected. After modal feature extraction, the respective graph modal features, acoustic modal features and biochemical modal features are obtained. Based on the graph modal features, acoustic modal features, and biochemical modal features of each observation point, a modal feature matrix is ​​constructed for each observation point. The modal inner product similarity is obtained for each observation point, and a modal covariance map is obtained for each observation point. After fusing the modal feature matrix and the modal covariance map for each observation point, flattening and dimensionality reduction operations are performed to obtain the respective ecological embedding representations. The node-based sea area spatiotemporal feature generation module is used to treat each observation point in the sea area to be detected as a node, connect the nodes with edges to obtain the sea area spatial graph structure; and perform graph convolution processing on the sea area spatial graph structure to obtain the spatial feature enhanced graph structure. In the spatial feature enhancement graph structure, the spatial structural features of each node are processed by temporal feature extraction to obtain the spatiotemporal features of the respective node's sea area. The biological community anomaly detection result generation module is used to obtain node anomaly values ​​based on the spatiotemporal characteristics of the node's sea area; and to obtain the node's anomaly level based on the mapping relationship between the node anomaly values ​​and the anomaly level standard. Based on the outliers of all nodes, a heatmap is constructed and fused with the geographic mapping to form a regional anomaly heatmap; the anomaly level of nodes is marked on the regional anomaly heatmap to obtain the anomaly detection results of the biological community.

6. A device for detecting abnormal changes in marine biological communities based on multimodal features, characterized in that, The system includes a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the method for detecting abnormal changes in marine biological communities based on multimodal features as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the method for detecting abnormal changes in marine biological communities based on multimodal features as described in any one of claims 1-4.

Images