Algae abundance dynamic feature selection method based on multi-agent reinforcement learning

Abstract

The invention discloses an algae abundance dynamic feature selection method based on multi-agent reinforcement learning, which belongs to the technical field of artificial intelligence and data mining, and comprises the steps of multi-source heterogeneous data acquisition and convergence, time sequence data cleaning and standardization processing, driving factor identification and weight calculation key driving factor set establishment, and dynamic feature selection. Convergence and determination of an optimal feature subset selected by multi-agent reinforcement learning and dynamic features: an MARLN system performs multi-round iteration and interactive learning, and each agent continuously optimizes a decision strategy thereof according to a global reward signal fused with a specification item; according to the method, the problems that a traditional static feature selection method cannot adapt to data changes and neglects interaction among features are solved, the most critical driving factors, namely the features, for algae abundance prediction are automatically recognized from multi-source heterogeneous data through a dynamic feature selection method, and the accuracy of algae abundance prediction is improved. And an optimal feature subset is constructed to improve the precision and robustness of the prediction model.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and data mining technology, and in particular to a method for dynamic feature selection of algal abundance based on multi-agent reinforcement learning. Background Technology

[0002] The core of algal abundance research is to analyze key factors affecting algal growth, combine mathematical models, sensor technology, or remote sensing methods to predict the trend of algal biomass changes in water bodies in advance, and ultimately provide a scientific support system for algal bloom control, water quality protection, and ecological management. The widespread application of sensor networks and the Internet of Things has generated massive amounts of multivariate time-series data. In the field of algal abundance prediction, directly using all the data for modeling faces a dual challenge: on the one hand, the computational and storage resources of the full data are heavy, and noise interference may cause key information to be buried; on the other hand, too many input variables can easily lead to the curse of dimensionality, resulting in a decrease in the model's generalization ability. Feature selection technology therefore becomes a crucial link. If the number of features is N, the goal is to select from features with a base of O(2^N)... N In the feature subset space of ), the optimal combination of variables is found to build an efficient prediction model.

[0003] Existing traditional feature selection techniques mainly include the Maximum Information Coefficient (MIC), Random Forest (RF), and Decision Tree (DT). Among them, MIC is a traditional mathematical statistical method, while RF and DT are machine learning methods.

[0004] However, traditional feature selection techniques have inherent limitations. First, the selection process is usually static; that is, after selecting a subset of features based on the initial training dataset, this subset remains fixed throughout the model's lifecycle. This static approach is ill-suited to scenarios where data distribution changes dynamically. Second, when evaluating features, traditional techniques often focus on measuring the independent relationship between a single feature and the prediction target, failing to adequately consider the complex temporal dependencies and interaction effects between features.

[0005] To address the shortcomings of static feature selection, dynamic feature selection techniques such as reinforcement learning and multi-agent systems have been introduced. These techniques involve designing agents that interact with the environmental dataset. The agents dynamically adjust their feature selection strategies based on feedback from model predictions, allowing the feature subset to adapt to changes in the data flow. However, existing reinforcement learning (RL) methods perform feature selection through mathematical modeling and data analysis, but do not adequately consider the actual biological drivers of algal growth. Simply relying on features such as data relevance and redundancy may lead to the inadvertent removal of some potentially crucial features for algal growth, thus affecting model accuracy and the understanding of algal growth mechanisms.

[0006] Secondly, while MARL can dynamically select features, it does not consider the correlation between features, which may lead to overfitting. Furthermore, in pursuit of high rewards, it may over-select certain features, resulting in overly sparse feature selection and neglecting other important features. Therefore, the field of algal abundance research urgently needs a new method that can ensure more accurate and balanced feature selection, improve the model's accuracy and generalization ability, and better reflect the actual causal relationship of algal growth. Summary of the Invention

[0007] The technical problem this invention aims to solve is to provide a dynamic feature selection method for algal abundance based on multi-agent reinforcement learning, which addresses the issues of traditional static feature selection methods failing to adapt to data changes and neglecting interactions between features. This method utilizes a triple multi-head self-attention mechanism to accurately identify and quantify key driving factors influencing algal abundance, enhancing agent reinforcement learning performance and improving algal abundance prediction capabilities. Through a dynamic feature selection method, the most critical driving factors (features) for algal abundance prediction are automatically identified from multi-source heterogeneous data, and an optimal feature subset is constructed to improve the accuracy and robustness of the prediction model.

[0008] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for dynamic feature selection of algal abundance based on multi-agent reinforcement learning, comprising the following steps:

[0009] S1. Multi-source heterogeneous data acquisition and aggregation: Using sensor networks to continuously acquire and detect various parameters of seawater in the sea area, and complete the aggregation of raw multi-source heterogeneous monitoring data;

[0010] S2. Time series data cleaning and standardization: Preprocessing and normalizing the original multi-source heterogeneous monitoring data stream to obtain a multivariate time series dataset;

[0011] S3. Driving Factor Identification and Weight Calculation: A Triple Multi-Head Self-Attention Mechanism (TSAM) model is constructed. The preprocessed multivariate time series dataset is input into the TSAM model, which deconstructs and analyzes the input data in parallel from two dimensions: water quality index parameters and time steps. Through parameter attention mechanism, time step attention mechanism, and parameter-time step cross-attention mechanism, complex nonlinear spatiotemporal dependencies are captured. Finally, a key index parameter-time step weight matrix is ​​output.

[0012] S4. Establishment of the set of key driving factors: Based on the "index parameter-time step weight matrix" generated in S3, the comprehensive influence of each environmental parameter at all time steps is statistically evaluated and ranked; the set of key driving factors that contribute most significantly and stably to the change in algal abundance is identified, and this set is used as the core feature pool to replace all the original parameters.

[0013] S5. Multi-agent Reinforcement Learning and Dynamic Feature Selection: A multi-agent reinforcement learning (MARLN) system is constructed. The set of key driving factors obtained in S4 is input into the multi-agent reinforcement learning system with a normalized term. Each driving factor is modeled as an independent agent. In each iteration, all agents make collaborative decisions based on the environmental state and their own policies to dynamically generate a feature subset. By introducing a correlation coefficient normalized term and a feature selection frequency normalized term into the reward function, the system intelligently balances the predictive power, low redundancy, and diversity of the feature subset, thereby avoiding overfitting and oversparseness problems.

[0014] S6. Convergence and Determination of Optimal Feature Subset: Through multiple rounds of iterative and interactive learning, the MARLN system continuously optimizes its decision-making strategy based on the global reward signal that incorporates the canonical term; and outputs the optimal feature subset.

[0015] S7. Algal Abundance Prediction and Result Visualization: Input the historical data corresponding to the optimal feature subset determined in S6 into the prediction model to perform accurate algal abundance prediction. The prediction results are displayed through a visualization interface, data interface, or report.

[0016] The technological advancements achieved by this invention, due to the adoption of the aforementioned technical solutions, are as follows: The TSAM model extensively utilizes the MHSA mechanism, which not only allows the information on the index parameters and time steps affecting the prediction results to be simultaneously mined and input into the model, but also identifies the driving factor matrix of the prediction results. This matrix can capture and quantify the degree of influence of a certain index parameter on the results at a specific point in time in the past. MARLN introduces a multi-agent system based on reinforcement learning and designs a reward function, introducing correlation coefficient normalization terms and feature selection frequency normalization terms from the perspectives of avoiding overfitting and excessive sparsity. Simultaneously, considering that agents cannot accurately determine whether reward fluctuations originate from environmental changes or noise caused by the exploration behavior of other agents, a denoising reward is introduced. This makes the selection process dynamic and improves prediction accuracy. The adoption of a denoising reward mechanism and an improved Q-value update strategy reduces interference from random exploration by multiple agents, improves learning efficiency and convergence stability, resulting in better stability. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a calculation diagram of the algal abundance dynamic feature selection method of the present invention;

[0019] Figure 2 This is a diagram of the Seq2Seq architecture that introduces a multi-head self-attention mechanism in this invention;

[0020] Figure 3 This is a structural diagram of the MHSA in this invention;

[0021] Figure 4 This is a structural diagram of the TSAM in this invention;

[0022] Figure 5 This is a schematic diagram of the MARLN interaction principle in this invention;

[0023] Figure 6 This is a graph showing the analysis results of algal growth driving factors in buoy 1 in this embodiment of the invention;

[0024] Figure 7 This is a graph showing the analysis results of algal growth driving factors in buoy 2 in this embodiment of the invention;

[0025] Figure 8 R is the experimental result in the embodiments of the present invention. 2 Bar chart comparison;

[0026] Figure 9 This is a comparison chart of RMSE histograms of experimental results in the embodiments of the present invention;

[0027] Figure 10 This is a comparison chart of MAE histograms of experimental results in the embodiments of the present invention. Detailed Implementation

[0028] The present invention will be further described in detail below with reference to embodiments:

[0029] like Figure 1 The diagram shows a flowchart of a dynamic feature selection method for algal abundance based on multi-agent reinforcement learning. The specific steps are as follows:

[0030] Step 1: Multi-source heterogeneous data acquisition and aggregation

[0031] The sensor network is used to continuously collect and detect various parameters of seawater in the sea area, and to complete the aggregation of raw, multi-source, heterogeneous monitoring data.

[0032] Step 2: Time-series data cleaning and standardization

[0033] The data preprocessing module receives the raw data stream, first performing outlier detection and labeling, and then imputing missing data to ensure sequence integrity. Next, the module downsamples and reassembles the high-frequency acquired raw sequences to generate a more stable time series. Finally, all feature sequences are normalized to a uniform numerical range, outputting a well-organized and standardized multivariate time series dataset.

[0034] Step 3: Identification of Driving Factors and Calculation of Weights

[0035] The preprocessed, standardized dataset was input into a Triple Multi-Head Self-Attention (TSAM) model. This model deconstructs and analyzes the input data in parallel from two dimensions: "water quality parameters" and "time steps." Through parameter attention, time step attention, and parameter-time step cross-attention mechanisms, it captures complex nonlinear spatiotemporal dependencies. The model ultimately outputs a crucial "parameter-time step weight matrix," which precisely quantifies the influence of each environmental parameter at each specific historical time point on the current algal abundance (chlorophyll a concentration). Details are as follows:

[0036] The TSAM module is the cornerstone of this invention. Its core function is to revolutionize traditional feature selection methods based on pure mathematics, accurately identifying and quantifying key driving factors that conform to the biological mechanisms of algal growth from the data. It receives raw multidimensional time-series data and outputs a "driving factor matrix" with clear ecological interpretation, providing a reliable and trustworthy feature pool for subsequent optimization.

[0037] This module is implemented based on the Triple Self-Attention Mechanism (TSAM) model.

[0038] The TSAM model directly aligns the input variables and time-based self-attention weights with the output, dividing the input data into a sequence of indicator parameters and a sequence of time steps. This allows the water quality indicator parameters and time-step self-attention mechanism in the model to be directly correlated with the model's output, resulting in an indicator parameter-time-step weight matrix, which represents the driving factors and reflects the degree of influence of a particular water quality indicator parameter on the prediction result at a specific time step. When predicting algal abundance, the TSAM model not only allows the model to obtain more variable information before outputting the results for a more balanced fit, but also avoids the problem of traditional time-series models failing to present the driving factors of the predicted values.

[0039] To more accurately identify the driving factors of algal abundance, conducting predictions with good fitting capabilities is a prerequisite. Since it is necessary to avoid the difficulty in interpreting driving factors due to temporal structure while achieving high prediction accuracy, this invention uses a sequence-to-sequence (Seq2Seq) architecture in the information extraction part. First, an encoder receives the input sequence and compresses it into a context vector, which captures all the information of the input sequence. Then, a decoder receives the context vector from the encoder and generates the output sequence based on this information. Simultaneously, this invention inputs the context vector output by the encoder into a multi-head self-attention (MHSA) mechanism to dynamically focus on different parts of the context vector, thereby avoiding the loss of contextual information; and MHSA performs a more balanced and comprehensive extraction of parameter and time step information. The Seq2Seq architecture incorporating MHSA is as follows: Figure 2 As shown. Multi-Head Self-Attention (MHSA) is an extension of the self-attention mechanism, allowing the model to process information in parallel across multiple subspaces, thereby enhancing the model's expressiveness and flexibility. The core idea of ​​MHSA is to divide the self-attention operation into multiple "heads," each of which computes attention in parallel across different subspaces. Therefore, each head can learn different attention patterns, enabling the model to better capture complex relationships in the input sequence. The outputs of these heads are then concatenated and a linear transformation is applied to obtain the final result. The MHSA structure is shown below. Figure 3 As shown.

[0040] The formula for calculating MHSA is:

[0041]

[0042] Where m is the number of "heads" in MHSA; concat means concatenating the outputs of each head. m The definition of is:

[0043]

[0044] In a self-attention mechanism, the Q-value (Query), K-value (Key), and V-value (Value) all originate from the same sequence. The similarity in a self-attention mechanism is calculated using a dot product scaling method.

[0045] Suppose the input is a sequence of length n, where each element is a d-dimensional vector: Each The input sequence is processed through three different linear transformations to compute the query, key, and value, respectively:

[0046]

[0047]

[0048]

[0049] in, , and This is a weight matrix obtained through learning. Next, the importance of each position is determined by calculating the similarity between the query and the key using a dot product:

[0050]

[0051] in, This is the dimension of the key, used for scaling to prevent the dot product value from becoming too large. Then, the attention score for each query location is normalized using the Softmax function to obtain the attention weights:

[0052]

[0053] in, This represents the attention strength of position i to position j in the input sequence, reflecting the degree to which position i depends on position j when calculating the output. Based on the attention weights, a weighted sum is calculated to obtain the final representation for each position:

[0054]

[0055] Step 4: Establishing the set of key driving factors

[0056] Based on the "index parameter-time step weight matrix" generated in step 3, the analysis module statistically evaluates and ranks the comprehensive influence of each environmental parameter across all time steps. According to preset screening criteria (such as the average attention weight threshold), a subset of parameters that contributes most significantly and stably to changes in algal abundance is identified from all input parameters and formally established as the set of key driving factors. This set serves as the core feature pool, replacing all the original parameters, and is input into the subsequent dynamic selection system.

[0057] The TSAM model structure is as follows: Figure 4 As shown,

[0058] Define the input sequence X That is, the input data contains N water quality index parameters, and the sequence length of each parameter is... Note that the input sequence X is essentially a matrix, so this matrix can be represented by rows and columns respectively, i.e. The sequence of indicator parameters is represented by columns: The time step sequence is represented row by row: The purpose of studying the driving factors of algal abundance is to... The influence of water quality parameter i on the final prediction result at time step j is analyzed, and the result of this driving factor is then used as the input for subsequent prediction studies.

[0059] For the sequence of index parameters, a multi-head self-attention mechanism is established to determine the relative contribution of different input water quality index parameters to the final result. The input is fed into a parameter encoder deep neural network (DNN) to obtain a more comprehensive and abstract parameter representation. To obtain the attention weights for each parameter at time step t, the attention weights will be... Input indicator parameter MHSA, set Let be the hidden state of the parameter decoder at time step t-1. , as well as All from The parameters of MHSA output can be obtained. for:

[0060]

[0061] If we assume that this parameter MHSA has There are *n* attention subspaces, and the attention weight matrix for each subspace is as follows: ,in The attention matrix for each subspace can be represented as:

[0062]

[0063] Next, output the MHSA parameter. The parameter is passed as input to the parameter decoder to obtain the new parameter decoder output. and :

[0064]

[0065] in, This represents the memory cell state of the parameter decoder at time step t-1. The input is fed into the parameter encoder at the next time step t+1, and simultaneously... The input is fed into the parameter decoder of t+1.

[0066] For time-step sequences, a multi-head self-attention mechanism is established to capture potential temporal dependencies in each parameter sequence. The input is fed into a stacked LSTM with a time-step encoder to obtain... To extract timing information. To obtain at time step j The attention matrix will Input into time step MHSA, set Let be the hidden state of the decoder at time step t-1. , as well as All from The time step MHSA output can be obtained. for:

[0067]

[0068] If we assume that MHSA has at this time step There are *n* attention subspaces, and the attention weight matrix for each subspace is as follows: ,in The attention matrix for each subspace can be represented as:

[0069]

[0070] Next, output the time step MHSA. The new time-step decoder output is passed as input to the time-step decoder. and :

[0071]

[0072] in, This represents the memory cell state of the time-step decoder at time step t-1. The input is fed into the time step encoder at the next time step t+1, and simultaneously... The input is fed into the time step decoder at time step t+1.

[0073] Considering that water quality parameter information and time series information contribute differently to the final prediction result, in order to obtain the influence of different driving factors combined with each parameter and time step on the final prediction result, the parameter decoder output is... and time step decoder output Perform the splicing operation to obtain h t A multi-head self-attention mechanism based on water quality indicator parameters and time steps was established, demonstrating that this mechanism effectively integrates the indicator parameter dimension and the time series dimension. , as well as All from We can obtain:

[0074]

[0075] The Output parameter is the output of the multi-head self-attention mechanism based on the indicator parameter and time step. This mechanism extracts the indicator parameter-time step relationship matrix M, where each element represents a driving factor—the contribution of a specific water quality indicator parameter to algal abundance changes at a particular time in the past. The Output parameter is then fed into a 1×1 convolution, with the number of input channels varied and the data nonlinearly combined to compress the information. Finally, the compressed data, containing complete indicator parameter-time step information, is input into a Linear layer to obtain the final prediction result.

[0076] To avoid overfitting, weight normalization is introduced in each MHSA module. Weight normalization limits the size of the weight matrix within a reasonable range, preventing overfitting to the input data and improving the model's generalization ability. Furthermore, weight normalization increases model stability and training speed, preventing gradient vanishing or exploding, thus accelerating network convergence. Let the weight matrix be W, which can be decomposed into the following form:

[0077]

[0078] Where v is a vector representing the direction of the weights; ||v|| is the Euclidean norm of vector v (i.e., its magnitude); g is a scalar representing the magnitude of the weights, which is a learnable parameter.

[0079] The above analysis shows that the TSAM model can not only comprehensively extract the time-series information of each water quality parameter and the interrelationships between different water quality parameters, but also has the ability to identify the driving factors of the prediction results. Therefore, this chapter analyzes the driving factors of algal abundance based on the TSAM model.

[0080] Step 5: Multi-agent reinforcement learning and dynamic feature selection

[0081] A multi-agent reinforcement learning (MARLN) system is constructed by inputting the set of key driving factors obtained in step 4 into the MARLN system with canonical terms. In the system, each driving factor is modeled as an independent agent. In each iteration, all agents collaboratively make decisions based on the environmental state (current feature selection status) and their own policies, dynamically generating a feature subset. This system intelligently balances the predictive power, low redundancy, and diversity of the feature subset by introducing correlation coefficient canonical terms and feature selection frequency canonical terms into the reward function, thereby avoiding overfitting and excessive sparsity problems. Specifically:

[0082] Dynamic Feature Selection Optimization Module (MARLN Module):

[0083] This module is the core of this application. It receives a high-quality feature pool from the TSAM module and transforms the static combinatorial optimization problem of feature selection, which ignores the interactions between features, into a dynamic, collaborative multi-agent decision-making process. By introducing a dual normalization term and a noise reduction mechanism, it outputs a feature subset that achieves optimal prediction performance, information diversity, and generalization ability.

[0084] This module is implemented using the Multi-Agent Reinforcement Learning with Normalizations (MARLN) module, and its interaction principle is as follows: Figure 5 As shown.

[0085] While traditional MARL can dynamically select features, it fails to consider overfitting due to the relationships between selected features, nor does it account for excessive sparsity caused by the tendency to repeatedly select one or a few features for higher rewards. Therefore, this invention introduces a normalization term into the feedback signal to normalize feature correlation and feature selection frequency from the perspectives of avoiding overfitting and excessive sparsity, respectively.

[0086] Normalizing feature correlation aims to penalize features with high correlation within a feature subset, encouraging the model to select feature combinations with relatively low correlation. This ensures that the correlation between selected features is within a reasonable range, guaranteeing that the selected features have higher informational value during learning. In other words, if the correlation between features in a subset is too high, the information contained in these features is almost identical. When selected simultaneously, the model may become too complex to adapt to new, unseen data, leading to overfitting. In this part, the invention uses the Pearson correlation coefficient method to represent the correlation between variables. The Pearson correlation coefficient method is a traditional filtering method with advantages such as strong versatility, and normalized weights are introduced into the correlation coefficient. In the reward function, the normalized correlation coefficient term is combined with task performance; therefore, the reward function is shown in the following equation:

[0087]

[0088] Where reward_cor is the reward output with a correlation coefficient normalization term; β is the weight coefficient obtained through training; These are the weights used for the correlation coefficient normalization term obtained through training; cor_Norm is the correlation coefficient normalization term, hence the calculation formula is:

[0089]

[0090] in, For matrix The value of the element in the k-th row and i-th column; The average value of the elements in the i-th column; The average value of the elements in the j-th column; It is the correlation coefficient between feature i and feature j.

[0091] The purpose of normalizing the feature selection frequency is to reduce the sparsity of feature selection by limiting the frequency with which the model selects features. Without penalizing the feature selection frequency, the feature subset might be biased towards selecting only a few features to obtain the maximum immediate reward, while ignoring other useful features. This could lead to extreme sparsity in feature selection, which is detrimental to comprehensive learning of the task. By normalizing the feature selection frequency, more features or a more balanced subset of features are selected, rather than just the few most frequently selected features, ensuring a more comprehensive consideration of available information, rather than relying solely on a small number of features. Similarly, normalization weights are introduced for the feature selection frequency, adding an additional normalization term to the reward function, hence the reward function is shown in the equation:

[0092]

[0093] Where reward_fre is the reward output with feature selection frequency specification; It is the weight of the feature selection frequency normalization term obtained through training; fre_Norm is the feature selection frequency normalization term, which represents the number of times a certain feature is selected in the entire feature selection process.

[0094] Considering that the number of features n in the selected subset is not necessarily equal to the maximum number of features K in the optimal subset, the final reward for:

[0095]

[0096]

[0097] Furthermore, in MARLN, when other agents perform exploratory actions, their actions may alter the current state of the environment or interfere with the feedback signal. In this case, the reward received by the current agent does not fully reflect its own actions but may contain noise from the exploratory actions of other agents. This makes it difficult for the agent to accurately determine the source of the reward, causing it to be unable to distinguish whether the fluctuation in reward comes from the environment (external factors) or the exploratory actions of other agents (internal factors). Therefore, the agent may mistakenly believe that its own actions generated the reward when in fact the reward fluctuation is caused by the actions of other agents. To eliminate this noise and make the feedback signal more accurately reflect the current agent's behavior, this study further introduces denoised rewards into the feedback signal.

[0098] The noise reduction reward is defined as the difference between the global reward a feature receives when it takes a random action while all other features take optimal actions, and the global reward if each feature takes optimal actions:

[0099]

[0100] in, This represents the total reward of the entire system when the optimal action is chosen for all features. This indicates that feature i has chosen a match with its current action. Converse random actions The remaining features continue to perform the optimal action, and the global reward for feature i is calculated. This indicates a noise reduction reward, which removes noise from the random behavior of other features, making the reward of this feature more realistically reflect its own behavior rather than the influence of other features.

[0101] According to Q-learning, the Q-value update includes a future reward discount term. Considering the noise introduced by the actions of other agents, the agent cannot accurately estimate future rewards. Therefore, the Q-value update focuses on the agent's immediate feedback rather than directly considering future rewards. Thus, the Q-value is updated as follows:

[0102]

[0103] The two modules work in tandem, forming a complete technical chain from "mechanistic safeguards" to "dynamic optimization." The TSAM module works first, acting as a "domain expert" to identify truly important factors from the raw data (generating a feature pool), ensuring the analysis starts correctly and is interpretable. Then, the MARLN module acts as an "efficient scheduler," dynamically combining the optimal team (optimal feature subset) from the reliable candidate list (feature pool) provided by the "expert" through learning and game theory among agents. This subset is used for algal abundance prediction, avoiding the computational burden and noise interference of using all parameters.

[0104] Step 6: Convergence and Determination of the Optimal Feature Subset

[0105] The MARLN system employs multiple rounds of iterative and interactive learning, with each agent continuously optimizing its decision-making strategy based on a global reward signal incorporating canonical terms. Ultimately, the joint policy of the agent ensemble converges to a stable state, at which point the feature combination they collectively choose is determined as the optimal feature subset. This subset represents the globally optimal solution found by the system after considering prediction performance, relationships between features, and long-term rewards.

[0106] Step 7: Algal Abundance Prediction and Result Visualization

[0107] The historical data corresponding to the optimal feature subset determined in step 6 are input into the prediction model to perform accurate prediction of algal abundance. The prediction results are displayed through visualization interfaces, data interfaces, or reports, providing direct and reliable data support.

[0108] Example 1:

[0109] Taking data from a national-level marine monitoring station in Beihai City, Guangxi Zhuang Autonomous Region, adjacent to the Beibu Gulf as an example, this station has deployed multiple buoys in the Beibu Gulf to monitor seawater quality parameters across the entire area. The buoys are equipped with solar-powered systems and YSI 6600 V2 multi-parameter water quality sensors, using GPRS wireless transmission technology to transmit data. They also feature safety protection and self-testing systems to effectively ensure stable operation of the observation points even under long-term unattended conditions. The sensors on the buoys include early blue protein sensors, chlorophyll a sensors, water temperature and conductivity sensors, turbidity sensors, pH sensors, dissolved oxygen sensors, and salinity sensors. Specific monitoring parameters of the sensors are shown in Table 1.

[0110] Table 1 Monitoring parameters of the buoy-mounted sensors

[0111]

[0112] Different detection parameters have varying degrees of influence on algal abundance, i.e., Chl-a concentration. Therefore, the dataset includes Sal, WT, pH, DO, EC, Turb, PE, and Chl-a data collected by a multi-parameter water quality sensor. Data for each water quality indicator was collected from the monitoring station over two years, from January 1, 2020 to December 31, 2021, with a sampling period of 30 minutes, totaling 35,040 samples.

[0113] Based on the algal growth drivers obtained from the TSAM model on the datasets of buoy 1 and buoy 2, as follows: Figures 6 to 7 As shown. Figure 6 and Figure 7 In the graph, the horizontal axis represents different variables, and the vertical axis represents different time points. Therefore, each element represents the degree of influence of a certain water quality indicator on algal growth at a specific time point. It can be observed that from t-1 to t-8, WT and Turb have significant effects; from t-9 to t-15, DO and pH have significant effects; Chl-a and PE have relatively balanced effects at different times; while Sal and EC contribute little to the prediction results over time from t-1 to t-15. Therefore, the main factors influencing algal growth differ at different times. However, overall, Chl-a, DO, PE, pH, WT, and Turb have significant effects and can be considered driving factors for algal growth, while Sal and EC have consistently small effects and can be considered non-driving factors. Furthermore, since the TSAM model can accurately predict and reliably identify driving factors, the six water quality indicators Chl-a, DO, PE, pH, WT, and Turb in the dataset are considered driving factors and treated as the original feature set, serving as input for subsequent feature selection studies.

[0114] In the process of using MARLN to find the optimal feature subset, each agent (feature) is encoded from 0 to 1, where 0 indicates that the agent is excluded and will not appear in the subset; and 1 indicates that the agent is included and is considered part of the optimal subset. In the study of driving factors, Chl-a, DO, PE, pH, WT, and Turb are the driving factors. Therefore, the original feature set in this part of the study consists of the other five parameters excluding Chl-a as the dependent variable. The size of the optimal subset K will greatly affect the feature selection process. Therefore, K is set to different values ​​from 1 to 5 for experiments. To illustrate the generality of feature selection, the commonly used sequence prediction model LSTM is used as the predictor. The hyperparameters include learning rate, hidden layer size, input sequence length, batch size, number of layers, dropout rate, activation function, optimizer, and number of training epochs, with values ​​of [0.001, 256, 64, 64, 2, 0.2, Relu, Adam, 300].

[0115] When the experimental locations are different, i.e., the datasets are buoy 1 and buoy 2 respectively, the experimental results are shown in Tables 2 and 3. To visually compare the performance of various methods, bar charts are plotted based on the data in Tables 2 and 3, as follows. Figure 8 , 9 As shown in Figure 10.

[0116] Table 2. Performance of different feature selection methods on the Buoy 1 dataset.

[0117]

[0118] Table 3. Performance of different feature selection methods on the Buoy 2 dataset.

[0119]

[0120] from Figures 8 to 10 It can be seen that when using MIC for feature selection, the results fluctuate greatly, indicating that traditional mathematical methods rely heavily on data and lack good generalization ability. RF and DT show good stability in feature selection, but still fail to achieve optimal performance. When the optimal subset size is small (K=1), MARLN performs similarly to other methods. As K changes, the performance of MARLN also improves, achieving good results. When K varies, MARLN not only exhibits very stable performance, but in most cases, its performance significantly outperforms other methods.

[0121] The results show that the optimal subset selected on the buoy 1 dataset is {WT, DO, Turb}, and the optimal subset selected on the buoy 2 dataset is {WT, DO}.

[0122] The TSAM model extensively utilizes the MHSA mechanism, which not only allows for the simultaneous extraction and input of both the indicative parameter information and time step information influencing the prediction results into the model, but also identifies the driving factor matrix of the prediction results. This matrix captures and quantifies the degree of influence of a specific indicator parameter on the result at a particular point in time. Regarding the interpretability of the MHSA mechanism, current research has clearly explained the information interaction within it, for example, by using attribution theory to construct attribution trees to illustrate the interactions at different levels within MHSA. This work lays the foundation for subsequent dynamic feature extraction.

[0123] MARLN introduces a multi-agent system based on reinforcement learning and designs a reward function, incorporating correlation coefficient normalization terms and feature selection frequency normalization terms to avoid overfitting and excessive sparsity. Considering that agents cannot accurately determine whether reward fluctuations originate from environmental changes or noise caused by the exploratory behavior of other agents, a noise reduction reward is introduced. To demonstrate the effectiveness of the proposed dynamic feature selection method, comparative experiments were conducted using traditional mathematical statistics and machine learning as contrasting methods. Experimental results show that MARLN can automatically find the optimal feature subset through interactive learning, while considering the relationships between features and the feedback of prediction results. The feature selection process can be adjusted, making it dynamic and improving prediction accuracy. Prediction results also show that using the optimal feature subset as input yields the best prediction performance. Compared to inputting all parameters into the model and inputting no parameters, R² increases by 0.0341 and 0.0667, respectively, proving that algal abundance changes are not only related to their own features but also influenced by external factors.

[0124] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A method for dynamic feature selection of algal abundance based on multi-agent reinforcement learning, characterized in that: The steps include the following: S1. Multi-source heterogeneous data acquisition and aggregation: Using sensor networks to continuously acquire and detect various parameters of seawater in the sea area, and complete the aggregation of raw multi-source heterogeneous monitoring data; S2. Time series data cleaning and standardization: Preprocessing and normalizing the original multi-source heterogeneous monitoring data stream to obtain a multivariate time series dataset; S3. Driving Factor Identification and Weight Calculation: A Triple Multi-Head Self-Attention Mechanism (TSAM) model is constructed. The preprocessed multivariate time series dataset is input into the TSAM model, which deconstructs and analyzes the input data in parallel from two dimensions: water quality index parameters and time steps. Through parameter attention mechanism, time step attention mechanism, and parameter-time step cross-attention mechanism, complex nonlinear spatiotemporal dependencies are captured. Finally, a key index parameter-time step weight matrix is ​​output. S4. Establishment of the set of key driving factors: Based on the "index parameter-time step weight matrix" generated in S3, the comprehensive influence of each environmental parameter at all time steps is statistically evaluated and ranked; the set of key driving factors that contribute most significantly and stably to the change in algal abundance is identified, and this set is used as the core feature pool to replace all the original parameters. S5. Multi-agent Reinforcement Learning and Dynamic Feature Selection: A multi-agent reinforcement learning (MARLN) system is constructed. The set of key driving factors obtained in S4 is input into the multi-agent reinforcement learning system with a normalized term. Each driving factor is modeled as an independent agent. In each iteration, all agents make collaborative decisions based on the environmental state and their own policies to dynamically generate a feature subset. By introducing a correlation coefficient normalized term and a feature selection frequency normalized term into the reward function, the system intelligently balances the predictive power, low redundancy, and diversity of the feature subset, thereby avoiding overfitting and oversparseness problems. S6. Convergence and Determination of Optimal Feature Subset: Through multiple rounds of iterative and interactive learning, the MARLN system continuously optimizes its decision-making strategy based on the global reward signal that incorporates the canonical term; and outputs the optimal feature subset. S7. Algal Abundance Prediction and Result Visualization: Input the historical data corresponding to the optimal feature subset determined in S6 into the prediction model to perform accurate algal abundance prediction. The prediction results are displayed through a visualization interface, data interface, or report.

2. The method for dynamic feature selection of algal abundance based on multi-agent reinforcement learning according to claim 1, characterized in that: The TSAM model in S3 has the following specific structure from input to output: encoder layer, MHSA layer, decoder layer, splicing layer, driver layer, and output layer.

3. The method for dynamic feature selection of algal abundance based on multi-agent reinforcement learning according to claim 2, characterized in that: The MHSA structure and specific operations are as follows: ； Where m is the number of "heads" in MHSA; concat means concatenating the outputs of each head. m The definition of is: ； Self-attention mechanism, namely Q-value, K-value, and V-value, where Q-value represents Query, K-value represents Key, and V-value represents Value, all three values ​​come from the same sequence; the similarity of self-attention mechanism is calculated in the form of dot product scaling; Suppose the input is a sequence of length n, where each element is a d-dimensional vector: X = [x1, x2, …, x…] n ], each The input sequence is processed through three different linear transformations to compute the query, key, and value, respectively: ； ； ； Among them, W q W k and W v The weight matrix is ​​obtained through learning; next, the importance of each position is determined by calculating the similarity between the query and the key using dot product. ； Where, d k The dimension is the key, used for scaling to prevent excessively large dot product values; then, the attention score for each query location is normalized using the Softmax function to obtain the attention weights: ； Where, α ij This represents the attention strength of position i to position j in the input sequence, reflecting the degree to which position i depends on position j when calculating the output; based on the attention weights, a weighted sum is calculated to obtain the final representation of each position: 。 4. The method for dynamic feature selection of algal abundance based on multi-agent reinforcement learning according to claim 3, characterized in that: To avoid overfitting, a weight normalization mechanism is introduced in each MHSA module; let the weight matrix be W, it can be decomposed into the following form: ； Where v is a vector representing the direction of the weights; ||v|| is the Euclidean norm of vector v, i.e., its magnitude; g is a scalar representing the magnitude of the weights, which is a learnable parameter.

5. The method for dynamic feature selection of algal abundance based on multi-agent reinforcement learning according to claim 2, characterized in that: The specific calculations for each layer of the TSAM model are as follows: Define the input sequence X The input data contains N water quality parameters, each with a sequence length of T. L The input sequence X is essentially a matrix, which can be represented by rows and columns, i.e., X = [x 1 , x 2 ,…, x N ] = [x1, x2, …, x TL ], where the index parameter sequence is represented by columns: x i = [x , x , …, x The time step sequence is represented row by row: x j = [x , x , …, x ] T The purpose of studying the driving factors of algal abundance is to understand x The influence of water quality parameter i on the final prediction result at time step j is analyzed, and the result of this driving factor is used as the input for subsequent prediction studies. For the sequence of index parameters, a multi-head self-attention mechanism is established to determine the relative contribution of different input water quality index parameters to the final result; x i The input is fed into a parameter encoder deep neural network (DNN) to obtain a more comprehensive and abstract parameter representation f. i To obtain the attention weights for each parameter at time step t, f i Input the index parameter MHSA, and set h_f t-1 Let Q be the hidden state of the parameter decoder at time step t-1. f V f and K f All come from [h_f t-1 , f i The MHSA output parameter can be obtained. f for: ； Suppose that the parameter MHSA has H f There are *n* attention subspaces, and the attention weight matrix for each subspace is A. hf Where hf = 1,2, …, H f The attention matrix for each subspace can be represented as: ； Next, output the MHSA parameter. f The parameter is passed as input to the parameter decoder to obtain the new parameter decoder output h_f t and c_f t : ； Among them, h_f t-1 The memory cell state of the parameter decoder at time step t-1; h_f t The input is fed into the parameter encoder at the next time step t+1, along with c_f t Input into the parameter decoder at t+1; For the time-step sequence, a multi-head self-attention mechanism is established to capture potential temporal dependencies in each parameter sequence; in x j The input is fed into a stacked LSTM with a time-step encoder to obtain s j To extract timing information; to obtain s at time step j j The attention matrix will s j Input into time step MHSA, let h_t t-1 Let Q be the hidden state of the time step decoder at time step t-1. t V t and K t All come from [h_t] t-1 , s j The time step MHSA output can be obtained. t for: ； Suppose that the time step MHSA has H t There are *n* attention subspaces, and the attention weight matrix for each subspace is A. ht Where ht = 1, 2, …, H t The attention matrix for each subspace can be represented as: ； Next, output the time step MHSA. t The output h_t is passed as input to the time-step decoder to obtain the new time-step decoder output. t and c_t t : ； Among them, h_t t-1 The memory cell state of the time step decoder at time step t-1; h_t t The input is fed into the time step encoder at the next time step t+1, along with c_t. t The input is fed into the decoder at time step t+1; Output h_f from the parameter decoder t and the time step decoder output h_t t Perform the splicing operation to obtain h t A multi-head self-attention mechanism based on water quality index parameters and time steps was established, which effectively integrates the index parameter dimension and the time series dimension; at this time, Q ft K ft and V ft All come from h t We can obtain: ； Output is the output of the index parameter-time step multi-head self-attention mechanism. In this mechanism, the index parameter-time step relationship matrix M can be extracted. The elements in this matrix represent the driving factors, that is, the contribution of a certain water quality index parameter to the change of algal abundance at a specific time in the past. Then, Output is input into a 1×1 convolution, the number of input channels is changed and the data is nonlinearly combined to compress the information. Finally, the compressed index parameter-time step information with complete information is input into the Linear layer to obtain the final prediction result.

6. The method for dynamic feature selection of algal abundance based on multi-agent reinforcement learning according to claim 1, characterized in that: The MARLN system in S5 introduces a dual canonical term and a noise reduction mechanism into the multi-agent reinforcement learning (MARL) module. This allows for the creation of a high-quality feature pool output by the TSAM module, transforming the static combinatorial optimization problem of feature selection, which ignores the interaction between features, into a dynamic and collaborative multi-agent decision-making process.

7. The method for dynamic feature selection of algal abundance based on multi-agent reinforcement learning according to claim 6, characterized in that: The introduced dual normalization term normalizes feature relevance and feature selection frequency from the perspectives of avoiding overfitting and excessive sparsity, respectively; as detailed below: The aim of normalizing feature correlation is to penalize features with high correlation in a subset of features. The Pearson correlation coefficient method is used to represent the correlation between variables, and normalized weights are introduced into the correlation coefficients. In the reward function, the normalized correlation coefficient term is combined with task performance; therefore, the reward function is as follows: ； Where reward_cor is the reward output with a correlation coefficient normalization term; β is the weight coefficient obtained through training; α pcor These are the weights used for the correlation coefficient normalization term obtained through training; cor_Norm is the correlation coefficient normalization term, hence the calculation formula is: ； Where, x ki Let X be a matrix s The value of the element in the k-th row and i-th column; The average value of the elements in the i-th column; The average value of the elements in the j-th column; cor_Norm ij It is the correlation coefficient between feature i and feature j; Normalizing the feature selection frequency aims to reduce the sparsity of feature selection by limiting the frequency of feature selection. Normalization weights are introduced for the feature selection frequency, and an additional normalization term is added to the reward function. Therefore, the reward function is as follows: ； Where reward_fre is the reward output with feature selection frequency normalization term; α pcor These are the weights used for the feature selection frequency normalization term obtained through training; fre_Norm is the feature selection frequency normalization term, representing the number of times a feature is selected during the entire feature selection process; Considering that the number of features n in the selected subset is not necessarily equal to the maximum number of features K in the optimal subset, the final reward R N-N for: ； 。 8. The method for dynamic feature selection of algal abundance based on multi-agent reinforcement learning according to claim 6, characterized in that: The denoising mechanism introduces a denoising reward into the feedback signal, specifically as follows: The denoising reward is defined as the difference between the global reward obtained by a feature when it takes a random action while all other features take optimal actions, and the global reward when each feature takes optimal actions: ； in, This represents the total reward of the entire system when the optimal action is chosen for all features. This indicates that feature i has chosen a match with its current action a. i The opposite random action c i The remaining features continue to perform the optimal action, and the global reward for feature i is C. i This indicates a noise reduction reward, which removes noise from the random behavior of other features, making the reward of this feature more realistically reflect its own behavior rather than the influence of other features; According to Q-learning, the update of the Q-value includes a future reward discount term. Considering the noise introduced by the actions of other agents, the agent cannot accurately estimate future rewards. Therefore, when updating the Q-value, we focus on the immediate feedback of the current agent and do not directly consider future rewards. Hence, the Q-value is updated as follows: 。

Images