A marine community monitoring method and system based on memory hybrid prototype and knowledge distillation

Abstract

The present invention relates to the technical field of marine biological community monitoring, and in particular to a method and system for monitoring marine biological communities based on memory hybrid prototypes and knowledge distillation. The method includes preprocessing the acquired multi-source heterogeneous monitoring data; constructing multimodal joint features using the preprocessed data; training a cross-modal teacher model using the multimodal joint features; training a student model based on historical marine ecological sequence data; performing enhanced knowledge distillation on the student model using the cross-modal teacher model; predicting marine ecological community time series using the trained student model; and detecting anomalies based on historical data and predicted data. Based on a breakthrough in the bottleneck of existing marine biological community monitoring technology, the present invention achieves multi-dimensional technical improvements and application value.

Description

Technical Field

[0001] The present invention relates to the technical field of marine biological community monitoring, and in particular to a marine biological community monitoring method and system based on memory hybrid prototype and knowledge distillation. Background Art

[0002] Existing marine biome monitoring technologies face significant bottlenecks in detection accuracy and performance overhead. Traditional monitoring systems often rely on single-modal data (such as optical images or sonar signals), facing multiple challenges in complex marine environments. Water turbidity poses a natural barrier to optical monitoring equipment. Suspended particles scatter light, blurring captured biome images and reducing the accuracy of feature extraction based on visual recognition algorithms. Lighting conditions severely restrict the effective monitoring period of optical equipment. In deep-sea or nighttime environments, insufficient light makes it difficult to capture biome outlines, forcing monitoring systems to use artificial light sources, which not only disturbs marine life but also introduces new optical interference. Sonar signal monitoring is also significantly affected by background noise. Natural and man-made noise, such as ship propellers and seafloor earthquakes, can easily drown out the subtle acoustic signatures of organisms. Statistics show that such interference results in a biome misidentification rate of up to 15%-30%.

[0003] In addition, existing marine biological community monitoring technologies have significant deficiencies in the deep integration of multimodal data and knowledge transfer. Traditional methods mostly rely on single-modal data or simple feature splicing, which makes it difficult to deal with noise interference and data heterogeneity problems in complex marine environments, and lacks effective use of professional ecological knowledge. The present invention proposes to construct a cross-modal teacher model by fusing a calibrated language model with privileged knowledge distillation technology, which can dynamically combine professional knowledge in marine science literature with real-time monitoring data to generate high-quality feature representations containing deep semantic associations. At the same time, the design of the subtractive cross-attention mechanism can optimize the heterogeneous characteristics of multimodal data, suppress redundant information and strengthen the fusion of complementary features, thereby improving the model's ability to extract multi-dimensional features of biological communities under complex conditions such as low light and high turbidity, and solving the problems of insufficient cross-modal association and weak anti-interference ability in traditional methods.

[0004] The difficulty of high-performance model adaptation during edge device deployment is another bottleneck of existing technologies. Due to the high complexity and large amount of computation, mainstream monitoring models often face the contradiction between long inference delay and high energy consumption when running on terminal devices such as ocean buoys and underwater robots, making it difficult to meet the needs of real-time monitoring and long-term endurance. The present invention uses a privileged knowledge distillation mechanism to lightweight the complex teacher model containing rich ecological knowledge, so that the student model can efficiently reproduce the core behavior of the teacher model, while significantly reducing the number of model parameters and computational overhead. This lightweight design is optimized in combination with the computing power characteristics of edge devices. While maintaining monitoring accuracy, it effectively improves the operating efficiency of the model on low-power terminals, breaks through the technical barriers between "model performance" and "device capabilities", and provides a feasible solution for the long-term continuous operation of marine monitoring equipment.

[0005] In terms of the integration of temporal dynamic features and ecological expertise, existing technologies lack in-depth modeling of the multi-scale evolution laws of marine biological communities, and do not fully utilize domain expertise to guide model learning. The present invention constructs a hybrid prototype learning mechanism based on reconstruction to extract normal prototypes and periodic prototypes from monitoring data at different time scales, which can effectively capture short-term fluctuations, long-term trends and periodic patterns in community dynamics. Combined with the ability of the calibrated language model to analyze professional knowledge in ecological literature, field knowledge such as species habits and environmental response mechanisms is integrated into the prototype construction process, so that the model can not only learn temporal features based on data-driven learning, but also use prior knowledge to enhance the understanding of complex dynamic scenarios, improve the accuracy of abnormal pattern recognition and trend prediction, and solve the problem of traditional models lacking knowledge guidance and insufficient generalization ability in temporal analysis. Summary of the Invention

[0006] In order to solve the above-mentioned problems, the present invention provides a marine biological community monitoring method and system based on memory hybrid prototype and knowledge distillation.

[0007] In a first aspect, the present invention provides a method for monitoring marine biomes based on memory hybrid prototypes and knowledge distillation, which adopts the following technical solutions:

[0008] A marine biome monitoring method based on memory hybrid prototypes and knowledge distillation, including:

[0009] Acquire multi-source heterogeneous monitoring data;

[0010] Perform data preprocessing on the acquired multi-source heterogeneous monitoring data;

[0011] Use preprocessed data to construct multimodal joint features;

[0012] Leveraging multimodal joint features for cross-modal teacher model training;

[0013] Train student models based on historical marine ecological series data;

[0014] Enhance knowledge distillation of student models using a cross-modal teacher model;

[0015] Use the trained student model to predict marine ecological community time series;

[0016] Perform anomaly detection based on historical and forecast data.

[0017] Furthermore, the data preprocessing of the acquired multi-source heterogeneous monitoring data includes denoising the original image using a bilateral filtering algorithm for bio-optical images, suppressing the scattering noise caused by water turbidity by the joint weighting of spatial proximity and pixel similarity; Perform short-time Fourier transform and pass it through the Hanning window function Cut off the local signal segment and generate the time-frequency matrix; for numerical data such as temperature and salinity Standardize the text data and use the SBERT model to transform the ecological relationship triple text Encoded into a 768-dimensional semantic embedding vector .

[0018] Furthermore, the cross-modal teacher model training is performed using the pre-processed data, including adding a learnable position encoding PE to the multimodal joint feature to obtain the initial state , which is then processed by CLMs to generate contextual embeddings, including corrected attention, layer normalization, and feedforward operations. Finally, the Transformer encoder PT encoder (·) is used to pass the text modality To reconstruct the true value in the time series mode and realize cross-modal reconstruction, where the CLMs LN and FFN The definition is as follows:

[0019]

[0020] in yes Pass the second floor LN After the output, and are learnable scaling and translation parameters, and denote the mean and standard deviation, respectively. Stands for element-wise multiplication.

[0021] Furthermore, the student model is trained based on historical marine ecological sequence data, including given historical data , the reverse embedding layer will Convert to a learnable matrix , to capture temporal dependencies across multiple variables, First, normalization is performed through reversible instances to mitigate the impact of distribution shift. Then, the normalized embedding representation It is then fed into a Transformer encoder TST encoder (·) to model the temporal dependencies between multiple variables, and the final output is To fuse time series and cross-variable features.

[0022] Furthermore, the cross-modal teacher model is used to enhance knowledge distillation of the student model, including aligning the attention map between the enhanced ET encoder (·) in the teacher network and the student time series TST encoder (·) to encourage the student model to imitate the teacher's behavior to achieve positive correlation distillation; aligning the embedding space of the teacher model and the student model to achieve feature distillation; combining the overall distillation loss with the correlation distillation loss With feature distillation loss , to guide the student’s learning process, the total knowledge distillation loss is defined as:

[0023]

[0024] in, and Is used to balance the correlation distillation loss With feature distillation loss Contributed hyperparameters.

[0025] Furthermore, the use of the trained student model to predict the marine ecological community time series includes using the student model for reasoning, wherein: is from the distilled student model The time series embedding representation of Input into the projection function to complete future predictions, expressed as:

[0026]

[0027] in For historical data input of marine ecological communities, Represents the prediction results of marine ecological communities, and is a learnable parameter, and the final output Normalization is performed and the prediction loss is adopted SL 1 Loss function: in, Represents the number of prediction samples.

[0028] Furthermore, the anomaly detection based on historical data and predicted data includes the following steps: and forecast data Splice along the time axis into original sequence data , divide the original sequence data into multiple subsequences of equal length , where each subsequence are all considered as an independent training time series. By extracting the fragment features of different scales in the normal marine community data sequence, a prototype set with size differences is constructed. In this case, the given sequence is divided into non-overlapping segments and the generated sequences are The Transformer encoder is embedded into the high-dimensional feature space to update the fragment prototype and use the updated fragment prototype Query sequence Perform reconstruction to obtain the reconstructed sequence and implement the update of the fragment prototype.

[0029] Furthermore, the anomaly detection based on historical data and forecast data also includes learning the period prototype of normal time series to model the characteristics of different period patterns, and transforming the time series into Convert to frequency domain to calculate the average amplitude value, select the first k amplitude values ​​for period division, and learn the period prototype for each variable. For each observed variable in the time series, Learning a Cycle Prototype , randomly initialized Finally, the prototype is updated by weighted aggregation of each periodic fragment to achieve periodic prototype update.

[0030] Furthermore, the anomaly detection based on historical data and predicted data includes integrating the data of the sequence after updating and reconstructing through the fragment prototype and the period prototype, respectively constructing the reconstruction loss and entropy loss, and at the same time based on the period prototype With query vector The distance between them is used to construct the periodic loss in the feature space; finally, the total loss function is obtained based on the reconstruction loss, entropy loss and periodic loss.

[0031] The second aspect is a marine biome monitoring system based on memory hybrid prototype and knowledge distillation, including:

[0032] The data acquisition module is configured to acquire multi-source heterogeneous monitoring data;

[0033] The preprocessing module is configured to perform data preprocessing on the acquired multi-source heterogeneous monitoring data;

[0034] The joint module is configured to construct multimodal joint features using the preprocessed data;

[0035] The teacher training module is configured to utilize multimodal joint features to perform cross-modal teacher model training;

[0036] The student training module is configured to train the student model based on historical marine ecological sequence data;

[0037] The knowledge distillation module is configured to perform enhanced knowledge distillation on the student model using a cross-modal teacher model;

[0038] The prediction module is configured to use the trained student model to perform marine ecological community time series prediction;

[0039] The anomaly detection module is configured to perform anomaly detection based on historical data and predicted data.

[0040] In a third aspect, the present invention provides a computer-readable storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, for a method of monitoring marine biological communities based on memory hybrid prototypes and knowledge distillation.

[0041] In a fourth aspect, the present invention provides a terminal device comprising a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded and executed by the processor to implement the marine biological community monitoring method based on memory hybrid prototype and knowledge distillation.

[0042] In summary, the present invention has the following beneficial technical effects:

[0043] Based on the breakthrough of the bottleneck of existing marine biological community monitoring technology, the present invention has achieved multi-dimensional technical improvement and application value. At the multimodal data processing level, the cross-modal teacher model constructed by the present invention uses the calibration language model and privileged knowledge distillation technology to deeply integrate the professional knowledge in marine science literature into real-time monitoring data. Through the dynamic optimization of heterogeneous data such as optical images and sonar signals by the subtractive cross-attention mechanism, the feature distortion problems caused by environmental interference such as water turbidity, insufficient light, and background noise are effectively suppressed, and the extraction accuracy of multi-dimensional features of biological communities in complex scenarios is greatly improved. Compared with traditional single modality or simple splicing methods, significant breakthroughs have been achieved in the accuracy of biological recognition and the integrity of community structure analysis, providing a solid foundation for the accurate interpretation of marine ecological data.

[0044] In the field of edge device deployment, the privileged knowledge distillation mechanism of the present invention successfully breaks the contradiction between "model performance" and "device capabilities." By efficiently migrating the core knowledge of the complex teacher model to the lightweight student model, the model's computational complexity and energy consumption are greatly reduced while maintaining monitoring accuracy. This innovation significantly shortens the inference latency of terminal devices such as ocean buoys and underwater robots when running monitoring models, and effectively improves the device's endurance. It not only meets the real-time monitoring needs under dynamic changes in marine life scenarios, but also provides a low-cost, sustainable solution for long-term, continuous marine ecological monitoring, effectively promoting the transformation of marine monitoring equipment from cloud dependence to edge autonomy.

[0045] In terms of integrating temporal dynamic features with ecological knowledge, a hybrid prototype learning mechanism accurately extracts normal and periodic prototypes from multi-scale time series data, and combines this with ecological expertise from calibrated language model parsing to construct a monitoring system with deep semantic understanding capabilities. This mechanism can effectively capture short-term fluctuations, long-term trends, and periodic patterns in the evolution of marine biomes. In complex dynamic scenarios such as red tide warnings and biological migration predictions, it significantly improves the accuracy of abnormal pattern recognition and trend prediction. This overcomes the shortcomings of traditional models in time series analysis, which lack knowledge guidance and generalization capabilities. It provides a scientific and reliable decision-making basis for applications such as marine ecological protection and fishery resource management, and realizes the transition of marine biome monitoring from data perception to knowledge-driven. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Schematic diagram of a marine biological community monitoring method based on memory hybrid prototype and knowledge distillation according to Example 1 of the present invention;

[0047] Figure 2 Schematic diagram of the prediction results of Example 1 of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described in detail below with reference to the accompanying drawings.

[0049] Example 1

[0050] Reference Figure 1 In this embodiment, a method for monitoring marine biological communities based on memory hybrid prototypes and knowledge distillation includes:

[0051] Acquire multi-source heterogeneous monitoring data;

[0052] Perform data preprocessing on the acquired multi-source heterogeneous monitoring data;

[0053] Use preprocessed data to construct multimodal joint features;

[0054] Leveraging multimodal joint features for cross-modal teacher model training;

[0055] Train student models based on historical marine ecological series data;

[0056] Enhance knowledge distillation of student models using a cross-modal teacher model;

[0057] Use the trained student model to predict marine ecological community time series;

[0058] Perform anomaly detection based on historical and forecast data.

[0059] Specifically:

[0060] S1. Data input and preprocessing,

[0061] The data input module of this invention integrates multi-source heterogeneous monitoring data, including optical images of marine organisms, high-frequency sonar echo signals, time-series data from environmental sensors, and structured knowledge from historical documents. To address the characteristics of different modal data and noise interference, the preprocessing phase uses multi-dimensional cleaning, feature fusion, and time-series alignment techniques to construct a standardized dataset suitable for large language model (LLM) pre-training and anomaly detection model training.

[0062] S1-1. Optical image processing,

[0063] First, the bilateral filtering algorithm is used to filter the original image Denoising is performed by jointly weighting spatial proximity and pixel similarity to suppress the scattered noise caused by water turbidity:

[0064]

[0065] in is the spatial neighbor kernel, is the pixel similarity kernel, i , j , k , l is the coordinate value, It is a hyperparameter that controls the kernel width. After denoising, semantic segmentation is performed through the U-Net network, and the pixel-level biological contour mask matrix is ​​output. ,in C is the number of category channels. Further global average pooling is used to obtain a high-dimensional visual feature vector :

[0066]

[0067] in For the c The channels are located in space ( i ,j ), The output vector c elements, concatenate the average values ​​of all channels into a visual feature vector :

[0068]

[0069] S1-2. Sonar signal processing,

[0070] Time domain echo signal Perform short-time Fourier transform and pass it through the Hanning window function Cut off the local signal segment and generate the time-frequency matrix:

[0071]

[0072] The window function satisfies:

[0073]

[0074] Where T is the window length, and the Mel frequency cepstral coefficients (MFCC) are extracted based on the time-frequency matrix. The linear frequency is first converted to Mel frequency through the Mel filter bank, and then the 13-dimensional basic features are obtained by discrete cosine transform. The first-order and second-order differences are calculated and then spliced ​​into a 39-dimensional voiceprint feature vector. :

[0075]

[0076] in Combine logarithmic energy compression with a Mel filter:

[0077]

[0078] in Representative m Mel filter weights, input Indicates the i Frame No. k frequency components, through the Mel filter bank Weighted sum, then take the logarithm Compress the dynamic range and then use discrete cosine transform (DCT) for dimensionality reduction:

[0079]

[0080] This patent sets 40 Mel filters, that is, M=40.

[0081] S1-3. Environmental parameter processing,

[0082] For numerical data such as temperature and salinity To standardize:

[0083]

[0084] in , are the mean and standard deviation of the training set, N is the total number of samples, Avoid denominator being zero and output normalized vector .

[0085] S1-4. Document knowledge coding,

[0086] In the data preprocessing stage, the present invention uses the SBERT model to transform the ecological relationship triple text Encoded into a 768-dimensional semantic embedding vector SBERT encodes text through a twin network architecture. First, the input text k The word segmentation is a token sequence, generated after being processed by the WordPiece word segmenter , and append the position code p and segment code s to form the input representation:

[0087]

[0088] To get the output vector of the first token , and generates the final semantic embedding through a linear mapping layer:

[0089]

[0090] in W The weight matrix of the mapping layer adjusts the expression space of semantic features by learning the weight relationship between different dimensions. b It is a bias vector that increases the fitting ability of the model and prevents the output from always being centered on the origin.

[0091] S1-5. Feature splicing,

[0092] The final generated semantic vector and visual features , sonar characteristics , environmental characteristics Splicing by dimension to form multimodal joint features :

[0093]

[0094] This process achieves the numerical representation of ecological knowledge and aligns it with multimodal monitoring data in the semantic space. It not only retains the original characteristics of biological morphology, acoustic signals, and environmental parameters, but also embeds domain knowledge such as species habits and ecological rules, providing a dual-driven input foundation of "data-knowledge" for the pre-training of subsequent large language models. This standardized input enables the model to learn both data distribution patterns and professional knowledge constraints during training, effectively improving the accuracy and interpretability of community structure analysis and abnormal pattern recognition in complex marine environments. It provides standardized input that integrates domain knowledge for subsequent models.

[0095] S2. Cross-modal teacher model training,

[0096] S2-1. Divide the data set,

[0097] Combine multimodal features F Divided into historical data and the true value .

[0098] S2-2. Correcting attention mechanism,

[0099] First, add a learnable position encoding PE to the multimodal joint feature to get the initial state :

[0100]

[0101] These initial state representations are then processed by CLMs to generate contextual embeddings. This process involves a series of transformations: rectified attention, layer normalization, and feedforward operations that gradually refine the word representations through multiple layers of the network:

[0102]

[0103] in Indicates the i Layer in application and The intermediate representation after D Represents the hidden dimension of the language model.

[0104] The patented design of a calibrated attention mechanism, FocusAtt, enhances masked multi-head self-attention (MMSA) for processing multimodal marine ecological data in large language models (LLMs). Traditional MMSA often struggles to distinguish the importance of cross-modal and intra-modal interactions, leading to data entanglement. The calibrated attention mechanism (top) strengthens intra-modal interactions by down-weighting cross-modal interactions (e.g., the interaction between the time series token "10" and the text token "were"). FocusAtt is formulated as follows:

[0105]

[0106] in, yes After the first LN Output after , superscript i Indicates the first i samples, that is, the index to distinguish different input data when processing multiple samples. represents the attention mechanism, which is obtained by querying and key calculate The query, key and value are respectively input Through linear transformation 、 and get. Represents the dimension of the key vector, used to scale the dot product attention scores to ensure numerical stability. The term adjusts the attention score by suppressing cross-modal interactions, i represents the first i row, corresponding to the first i The position index of a token.

[0107] CLMs LN and FFN The definition is as follows:

[0108]

[0109] in yes Pass the second floor LN The output after. and are learnable scaling and translation parameters, and denote the mean and standard deviation, respectively. Stands for element-wise multiplication.

[0110] The output of the CLM module is recorded as and ,Then the embedding representation of the last token is extracted from these outputs.,Thanks to the masked attention mechanism in LLMs, the tokens at the end of the,reminder are able to aggregate the most comprehensive knowledge.,Specifically, the representation of the last token at a given position is,only influenced by its preceding tokens.

[0111] In order to utilize this feature and reduce computational overhead to achieve efficient knowledge distillation, we and Extract the last token embedding from: and This strategy preserves core information for subsequent distillation while ensuring computational efficiency.

[0112] S2-3. Negative cross attention,

[0113] A Negative Cross-Attention (NCA) mechanism is designed to eliminate the textual information embedded in the last token representation, ensuring that the retained embedding is still highly relevant for time series prediction.

[0114] In NCA, we first calculate the true value and historical data Projection embedding application layer normalization and projection function 、 and Then, the inter-channel similarity matrix is ​​calculated by matrix multiplication :

[0115]

[0116] in Represents matrix multiplication.

[0117] Then, by and Multiply by to perform channel-level feature aggregation. and This intersection is subtracted from the original data, and then processed by layer normalization and feedforward layer to obtain the refined true prompt embedding :

[0118]

[0119] in Represents the refined true embedding vector. Here is a linear layer, represents the subtraction operation. NCA refines the ground-truth cue embeddings and preserves the privileged knowledge for subsequent distillation. In addition, to avoid repeated processing with frozen CLMs, we store the embeddings after subtraction for efficient reconstruction.

[0120] S2-4. Cross-modal reconstruction,

[0121] This task uses the Transformer encoder PT encoder (·) to To reconstruct the true value of the time series mode. Inside the PT encoder (·), First in Layers undergo layer normalization deal with:

[0122]

[0123] in represents the normalized intermediate embedding. The structure is the same as before.

[0124] Normalized embedding Then, the multi-head attention layer (denoted as ) for processing. Output result Combined with the input via a residual connection:

[0125]

[0126] in 、 、 and is a learnable linear projection. The attention mechanism calculates the dependencies between feature dimensions.

[0127] Output Then through another layer Normalized and then passed through the feedforward network The final result is combined with the input through a residual connection:

[0128]

[0129] in, Represents the output of the PT encoder (·), which is denoted as ,and The structure is the same as before.

[0130] The reconstructed time series is then Applying the projection function generates:

[0131]

[0132] in represents the reconstructed time series, and is a learnable parameter. The reconstruction loss uses smooth L1 loss ( )definition:

[0133]

[0134] in, G Represents the total number of reconstruction samples. SL 1 The loss function ensures robustness to external outliers while maintaining sensitivity to small errors.

[0135] S3. Student model training,

[0136] The student model processes historical marine ecological series data through reversible instance normalization, an inverse embedding layer, a time series transformer, and a projection layer. Given historical data , the reverse embedding layer will Convert to a learnable matrix , to capture temporal dependencies across multiple variables. First, normalization is performed using reversible instances to mitigate the impact of distribution shift. The normalized data is then embedded as follows:

[0137]

[0138] in, represents the output of the inverse embedding layer. T represents the hidden dimension of the embedding, and and is a learnable parameter.

[0139] Embedded representation It is then fed into a Transformer encoder TST encoder (·), which is used to model the temporal dependencies between multiple variables. The encoder consists of multiple layers, each of which follows the following processing flow:

[0140]

[0141] in represents the intermediate embedding after layer normalization, and It is the output after combining with the input through the residual connection. Multi-head self-attention mechanism in the encoder Compute the attention-weighted representation:

[0142]

[0143] in 、 、 and is the learnable projection matrix, is the dimension of the key vector. The final output from the TST encoder (·) The time series and cross-variable features are integrated and these features will be used for downstream tasks.

[0144] S4. Strengthen knowledge distillation,

[0145] Enhanced knowledge distillation focuses on transferring privileged representations from a cross-modal teacher model to a lightweight student model through relevance and feature distillation.

[0146] S4-1. Correlation Distillation,

[0147] Relevance Distillation forces the student model to imitate the teacher’s behavior by aligning the attention maps between the reinforcement transformer (ET encoder (·)) and the student time series transformer (TST encoder (·)) in the teacher network. The teacher’s corrected attention map guides the construction of the student’s attention distribution to maintain the correlation dependencies between features. Specifically, the attention matrix obtained from the ET encoder (·) is and obtained from the TST encoder (·) , are averaged over all attention heads in the last encoder layer to form a unified representation. The correlation distillation loss is defined as:

[0148]

[0149] in, Ensure that students reduce sensitivity to outliers while replicating the teacher's contextual understanding of feature characteristics.

[0150] S4-2. Feature distillation,

[0151] Feature distillation aims to align the embedding spaces of the teacher model and the student model. The teacher model generates privileged embeddings through its ET encoder (·) , while the student model This process is replicated through the TST encoder (·) of its time series Transformer to produce an embedding The feature distillation loss also uses smoothing SL 1 Loss function implementation:

[0152]

[0153] This process ensures that the student model effectively captures the rich features learned by the teacher model, thereby achieving robust and accurate knowledge transfer.

[0154] The overall distillation loss combines the correlation distillation loss With feature distillation loss , to guide the student’s learning process. The total knowledge distillation loss is defined as:

[0155]

[0156] in, and Is used to balance the correlation distillation loss With feature distillation loss Contributed hyperparameters.

[0157] S5. Time series prediction of marine ecological communities,

[0158] (1) The time series prediction module uses a well-trained student model for efficient prediction. During testing, only the student model is used for inference. Specifically, is from the distilled student model The time series embedding representation of Input into the projection function to complete future prediction. Its mathematical expression is as follows:

[0159]

[0160] in For historical data input of marine ecological communities, Represents the prediction results of marine ecological communities. and is a learnable parameter. The final output Will be normalized. The prediction loss is SL 1 Loss function:

[0161]

[0162] in, Represents the number of prediction samples. The loss function of the proposed TimeKD method consists of four parts: reconstruction loss , correlation distillation loss , feature distillation loss and predicted losses We combine these losses to get the overall loss function as shown below:

[0163]

[0164] in, 、 and is a hyperparameter used for balancing.

[0165] (2) Processing of marine biome data series,

[0166] This patent achieves dynamic assessment of community status by integrating anomaly detection with historical and real-time data, and builds an ecological early warning mechanism based on anomaly detection in predicted data. Specifically, the system first establishes a normal pattern baseline for historical observation data, then detects abnormalities in the current community status through real-time data stream analysis. Finally, it combines the degree of deviation from the predicted data to provide advanced early warning of ecological risks.

[0167] First, historical data on marine biomes and forecast data Splice along the time axis into original sequence data :

[0168]

[0169] Afterwards, the original sequence data is divided into multiple subsequences of equal length , where each subsequence are all considered as an independent training time series. ,in L is the sequence length, Indicates time t The goal of the marine biome monitoring system is to learn knowledge from known normal data sequences and to predict unknown data sequences. Generate status label ,in , 0 means normal and 1 means abnormal. For the reconstruction-based model, the anomaly score is calculated by:

[0170]

[0171] in Reconstructed through learning knowledge Observed values, Indicates the L2 norm. Finally, the abnormal label is determined by the threshold Judgment: If but ,otherwise .

[0172] S6. Marine biome fragment prototype learning,

[0173] To capture multi-scale temporal information, the model extracts fragment features at different scales from a normal marine biome data sequence and constructs a set of prototypes with varying sizes. This method encodes local information from different time spans into prototypes of corresponding sizes, thus embedding a multi-perspective representation of normal features.

[0174] S6-1. Average pooling,

[0175] Given input data , divide it into several lengths Non-overlapping segments of The values ​​are averaged according to the formula:

[0176]

[0177] in The new sequence generated is ,in In particular, when z =1, That is the original time series X, the original information is retained after the pooling layer with a kernel size of 1. Through the average pooling operation, different z Value Xᶻ It can capture local information of different time spans and thus learn temporal dependencies of different durations.

[0178] S6-2. Update the fragment prototype,

[0179] To learn prototypes in feature space, the generated sequence Embedded into high-dimensional feature space through Transformer encoder:

[0180]

[0181] in , used to learn segment prototypes of different scales. The Transformer encoder embeds sequences of different scales into a high-dimensional feature space to learn feature prototypes. The following explains how the encoder works:

[0182]

[0183] Specifically, the pooled sequence First, a linear layer is used to embed the model into a high-dimensional space to represent more complex feature information and potential patterns. Then, a multi-layer Transformer module is used to capture long-term temporal dependencies, and finally a query sequence is generated for learning multi-scale segment prototypes. This feature will be fed into the time period query update module to learn the corresponding time period prototype for each variable. z In order to present differentiated variable association characteristics, this architecture configures an encoder separately for each time period.

[0184] Randomly initialized fragment prototypes Based on the query sequence Update Gate Update the fragment prototype With query sequence The similarity matrix between It can be calculated as follows:

[0185]

[0186] in is the temperature parameter. It should be noted that the fragment prototype not only needs to contain new information but also retain historical information. Therefore, the proposed update gate To update , and its update formula is:

[0187]

[0188] Where "◦" represents element-wise multiplication operation, The calculation formula is:

[0189]

[0190] in is the sigmoid activation function. and It is a randomly initialized learning matrix used to adjust the degree of retention and removal of historical prototypes and new information.

[0191] S6-3. Query Reconstruction

[0192] During the reconstruction process, the fragment prototype needs to effectively reconstruct the query sequence at each scale. z Next, use the updated fragment prototype Query sequence Reconstruct and obtain the reconstructed sequence :

[0193]

[0194] Represents a query Fragment prototype The attention weight matrix.

[0195] Reconstructed query sequences at different scales Appropriate integration is required to reconstruct the model considering different time dependency lengths. . Review the average pooling strategy, The local information is contained in Among them In other words, Under scale, Include Therefore, it is natural to consider ( j =1,2,..., m ) to reconstruct ， All related Reconstruction through neural network integration :

[0196]

[0197] Finally, the sequence will be reconstructed With the original scale characteristics Splice and restore to the original space through normal fragment prototype decoding , get the input sequenceX The reconstruction result .

[0198] S7. Study of the prototype of the marine biome cycle,

[0199] Most real-world time series have multiple periods, which influence each other and together represent the overall trend of the time series. In addition to conventional segment prototypes, this patent also models the characteristics of different periodic patterns by learning the period prototypes of normal time series.

[0200] S7-1. Cycle division,

[0201] The period division of a time series depends on the frequency information of the series in the frequency domain. To this end, the time series is transformed into Convert to the frequency domain and calculate the average amplitude using the following formula:

[0202] .

[0203] Considering the sparsity of the frequency domain, only the front k Amplitude value Used for period division, where:

[0204]

[0205] argTopK( ) indicates the selection before k amplitude, given period p ∈ P , time series X Can be divided into segments (with zero padding at the end). Since each observed variable has a unique variation cycle,

[0206] It is necessary to learn the period prototype for each variable separately. Extract univariate series from , which is divided into N The length is p and reshape it into . Will eventually X Reorganized into Prototype with learning cycle.

[0207] S7-2. Update cycle prototype,

[0208] Through the Transformer encoder Embedded For each observed variable of the time series, the period is divided into Learning a Cycle Prototype , where random initialization After that, the prototype is updated by weighted aggregation of each period fragment:

[0209]

[0210] in:

[0211]

[0212] in and is a learnable matrix. Finally, we get C The periodic prototype set of variables , used for query sequence reconstruction.

[0213] S7-3. Restructure the query

[0214] Query reconstruction. Considering the correlation between variables, each query vector Available through updated cycle prototypes Reconstruct and get , and its calculation formula is:

[0215]

[0216] Divide the reconstructed cycle into segments and Splicing Generation , where each . Summarize all C After the reconstruction of the variables is segmented, the decoder Refactored to , indicating the i The reconstruction data obtained in the final cycle is .

[0217] S8. Data Integration

[0218] After completing the reconstruction through the fragment prototype and the cycle prototype, the final and Perform weighted summation to reconstruct X get :

[0219]

[0220] in is a hyperparameter.

[0221] S9. Marine biome anomaly detection,

[0222] S9-1. Loss function and anomaly score,

[0223] Typically, during the training phase, the reconstruction loss is undoubtedly a component of the loss function, which is formulated as:

[0224]

[0225] in, represents the Frobenius norm. In addition, an excessive number of prototypes used for reconstruction may over-interpret normal information in the training time series. To ensure that only the most relevant prototypes are presented during reconstruction, it is necessary to impose a sparsity constraint on the reconstruction weights to reduce the possibility of overfitting. In this paper, entropy loss is used as a sparsity constraint on the reconstruction weights:

[0226]

[0227] Because the reconstruction process requires the participation of all periodic prototypes. Specifically, the periodic prototype must describe the periodicity of the training time series as appropriately as possible. Therefore, based on the periodic prototype With query vector The distance between them, the periodic loss in the feature space is designed:

[0228]

[0229] in, Representation feature space Middle i variables, and the period is p No. j query vectors. In summary, the total loss function LOSS for , and A weighted combination of:

[0230]

[0231] in, 、 and are adaptive parameters for different loss terms.

[0232] In order to detect anomalies, significant deviations in the input space and feature space are designed as anomaly scores. In the input space, the reconstruction error is usually regarded as an anomaly score. Finally, the value is calculated by a preset threshold. Determine the sample category:

[0233]

[0234] in yes At the moment t The specific value of y =1 indicates abnormality,y =0 indicates normal, and the accurate identification of abnormal points in the marine biological community data sequence is achieved by integrating multi-dimensional deviations.

[0235] Experimental verification:

[0236] Table 1 Comparison of monitoring effects

[0237]

[0238] Comprehensive analysis results show that the proposed method has significant advantages in both prediction accuracy and stability. Figure 2 shown.

[0239] The predicted curves maintain a high degree of consistency with the true values ​​in terms of overall trends. Especially during critical periods of significant data fluctuation, this method accurately tracks the actual trend, avoiding the prediction lag and oversmoothing often seen in baseline models. This excellent fit is fully demonstrated in the quantitative metrics shown in Table 1: Precision (a measure of the accuracy of predicted positive examples) improved by 7.9%, Recall (a measure of the ability to detect true positive examples) increased by 8.5%, and the overall evaluation metric F1-score (the harmonic mean of precision and recall) reached 92.76%, an 8.3% improvement over the optimal baseline model.

[0240] It is worth noting that the simultaneous improvement of the three indicators reveals the advantages of the method in many aspects: the improvement of precision indicates that the false alarm rate of the prediction results is significantly reduced; the greater increase in recall indicates that the method has a stronger ability to capture positive samples; the combined effect of the two results in a balanced improvement in the F1 value. Figure 2 The performance is that the prediction curve can respond to the sudden change of the true value in time (high recall rate characteristic) without generating false fluctuations (high precision rate characteristic).

[0241] Experiments also show that this method maintains stable performance across diverse data distribution scenarios. As shown in Table 1, even in the extreme case of insufficient training data (HMAP dataset), the F1 score remains above 0.90, surpassing the baseline model's 0.89. This effectively overcomes the overfitting vulnerability of traditional models in small sample sizes.

[0242] Example 2

[0243] This embodiment provides a marine biome monitoring system based on memory hybrid prototypes and knowledge distillation, including:

[0244] The data acquisition module is configured as follows:

[0245] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, a method for monitoring marine biological communities based on memory hybrid prototypes and knowledge distillation.

[0246] A terminal device includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded and executed by the processor. A method for monitoring marine biological communities based on memory hybrid prototypes and knowledge distillation is described.

[0247] The above are all preferred embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A marine biome monitoring method based on memory hybrid prototype and knowledge distillation, characterized in that: include: Acquire multi-source heterogeneous monitoring data; Perform data preprocessing on the acquired multi-source heterogeneous monitoring data; Use preprocessed data to construct multimodal joint features; Leveraging multimodal joint features for cross-modal teacher model training; Train student models based on historical marine ecological series data; Enhance knowledge distillation of student models using a cross-modal teacher model; Use the trained student model to predict marine ecological community time series; Anomaly detection based on historical and predicted data; The data preprocessing of the acquired multi-source heterogeneous monitoring data includes denoising the original image using a bilateral filtering algorithm for bio-optical images, suppressing the scattering noise caused by water turbidity by the joint weighting of spatial proximity and pixel similarity; Perform short-time Fourier transform and pass it through the Hanning window function Cut off the local signal segment and generate the time-frequency matrix; for temperature and salinity numerical data Conduct standardization processing; The SBERT model is used to transform the ecological relationship triple text into Encoded into a 768-dimensional semantic embedding vector ; The cross-modal teacher model training is performed using the pre-processed data, including adding a learnable position encoding PE to the multimodal joint feature to obtain the initial state , which is then processed by CLMs to generate contextual embeddings, including corrected attention, layer normalization, and feedforward operations. Finally, the Transformer encoder PT encoder (·) is used to pass the text modality To reconstruct the true value in the time series mode and realize cross-modal reconstruction, where the CLMs LN and FFN The definition is as follows: in yes Pass the second floor LN After the output, and are learnable scaling and translation parameters, and denote the mean and standard deviation respectively, Represents element-wise multiplication; The student model is trained based on historical marine ecological sequence data, including given historical data , the reverse embedding layer will Convert to a learnable matrix , to capture temporal dependencies across multiple variables, First, normalization is performed through reversible instances to mitigate the impact of distribution shift. Then, the normalized embedding representation It is then fed into a Transformer encoder TST encoder (·) to model the temporal dependencies between multiple variables, and the final output is To fuse time series and cross-variable features; The cross-modal teacher model is used to enhance knowledge distillation of the student model, including aligning the attention map between the enhanced ET encoder (·) in the teacher network and the student time series TST encoder (·) to encourage the student model to imitate the teacher's behavior to achieve positive correlation distillation; aligning the embedding space of the teacher model and the student model to achieve feature distillation; the overall distillation loss is combined with the correlation distillation loss. With feature distillation loss , to guide the student’s learning process, the total knowledge distillation loss is defined as: in, and Is used to balance the correlation distillation loss With feature distillation loss Contributed hyperparameters; The method of using the trained student model to predict the time series of marine ecological communities includes using the student model for reasoning, wherein: is from the distilled student model The time series embedding representation of Input into the projection function to complete future predictions, expressed as: in For historical data input of marine ecological communities, Represents the prediction results of marine ecological communities, and is a learnable parameter, and the final output Normalization is performed and the prediction loss is adopted SL 1 Loss function: in, Represents the number of prediction samples.

2. The method for monitoring marine biological communities based on memory hybrid prototypes and knowledge distillation according to claim 1, characterized in that: The anomaly detection based on historical data and predicted data includes the historical data of marine biological communities. X h and forecast data X f Splice along the time axis into original sequence data , divide the original sequence data into multiple subsequences of equal length , where each subsequence are all considered as an independent training time series. By extracting the fragment features of different scales in the normal marine community data sequence, a prototype set with size differences is constructed. In this case, the given sequence is divided into non-overlapping segments and the generated sequences are Xᶻ The Transformer encoder is embedded into the high-dimensional feature space to update the fragment prototype and use the updated fragment prototype Bᶻ Query sequence Qᶻ Perform reconstruction to obtain the reconstructed sequence and implement the update of the fragment prototype.

3. The method for monitoring marine biological communities based on memory hybrid prototypes and knowledge distillation according to claim 2, characterized in that: The anomaly detection based on historical data and forecast data also includes learning the period prototype of normal time series to model the characteristics of different period patterns, and transforming the time series into Convert to frequency domain to calculate the average amplitude value, select the first k amplitude values ​​for period division, and learn the period prototype for each variable. For each observed variable in the time series, Learning a Cycle Prototype , randomly initialized Finally, the prototype is updated by weighted aggregation of each periodic fragment to achieve periodic prototype update.

4. The method for monitoring marine biological communities based on memory hybrid prototypes and knowledge distillation according to claim 3, characterized in that: The anomaly detection based on historical data and predicted data includes integrating the data of the sequence after updating and reconstructing through the fragment prototype and the period prototype, respectively constructing the reconstruction loss and entropy loss, and at the same time, based on the period prototype With query vector The distance between them is used to construct the periodic loss in the feature space; finally, the total loss function is obtained based on the reconstruction loss, entropy loss and periodic loss.

5. A marine biological community monitoring system based on memory hybrid prototypes and knowledge distillation, which implements the marine biological community monitoring method based on memory hybrid prototypes and knowledge distillation according to claim 1, characterized in that: include: The data acquisition module is configured to acquire multi-source heterogeneous monitoring data; The preprocessing module is configured to perform data preprocessing on the acquired multi-source heterogeneous monitoring data; The joint module is configured to construct multimodal joint features using the preprocessed data; The teacher training module is configured to utilize multimodal joint features to perform cross-modal teacher model training; The student training module is configured to train the student model based on historical marine ecological sequence data; The knowledge distillation module is configured to perform enhanced knowledge distillation on the student model using a cross-modal teacher model; The prediction module is configured to use the trained student model to perform marine ecological community time series prediction; The anomaly detection module is configured to perform anomaly detection based on historical data and predicted data.

Images