Marine AI Analysis and Marine Disaster Prediction Method and System Based on Multi-Source Heterogeneous Data
By adopting AI analysis methods of multi-source heterogeneous data in marine disaster prediction, combining data dimensionality reduction and distributed neural networks, a multi-level state space and hierarchical early warning threshold are established, and a more efficient and accurate marine disaster prediction and early warning threshold are solved in the existing technology, and a more efficient and accurate marine disaster prediction and early warning are achieved.
Patent Information
- Application Number
- CN202510436258.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-06-17
- Estimated Expiration
- 2045-04-09
AI Technical Summary
Existing marine disaster prediction methods rely on a single data source and cannot make full use of multi-source data, resulting in insufficient prediction accuracy and untimely early warning response.
Using a marine AI analysis method based on multi-source heterogeneous data, a multi-level state space and a hierarchical warning threshold set are established through data matrix construction, information calculation, dimensionality reduction feature extraction, distributed neural network training and Markov decision-making algorithm to realize real-time abnormal state prediction and dynamic warning decision-making.
It improves the accuracy of marine AI analysis and marine disaster prediction, enhances the system's adaptability to dynamic changes in the marine environment, reduces the false alarm rate, and achieves faster and more accurate early warning response.
Smart Images

Figure CN119939176B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of multi-source heterogeneous data processing, and particularly relates to a method and system for ocean AI analysis and ocean disaster prediction based on multi-source heterogeneous data. Background Art
[0002] The multi-source heterogeneous data generated by ocean environmental monitoring systems has characteristics such as high dimensionality, multiple types, and strong real-time nature. Traditional data analysis methods are difficult to effectively process these complex data characteristics. Existing ocean disaster prediction methods mainly rely on a single data source for analysis, and cannot make full use of the rich information contained in multi-source data, resulting in insufficient prediction accuracy and untimely early warning responses.
[0003] The main challenges faced by current ocean monitoring data analysis are how to extract key features from massive heterogeneous data and how to build an accurate prediction model. Although deep learning methods have made significant progress in the field of data analysis, there are still problems such as high computational complexity and weak model generalization ability when dealing with ocean multi-source heterogeneous data, making it difficult to meet the requirements of real-time early warning. In view of the fast propagation characteristics of ocean disasters, a real-time early warning system needs to consider both prediction accuracy and response timeliness. Existing early warning methods often use fixed thresholds for judgment, lack the ability to adapt to the dynamic changes of the ocean environment, and are prone to false negatives or false positives. In addition, different levels of ocean disasters require different early warning strategies, and how to establish a reasonable multi-level early warning mechanism is also an urgent problem to be solved. Summary of the Invention
[0004] The main object of the present invention is to provide a method and system for ocean AI analysis and ocean disaster prediction based on multi-source heterogeneous data, and the present invention improves the accuracy of ocean AI analysis and ocean disaster prediction.
[0005] To achieve the above object, the present invention provides a method for ocean AI analysis and ocean disaster prediction based on multi-source heterogeneous data, including the following steps:
[0006] Construct a data matrix and calculate the information amount for ocean monitoring data to obtain a monitoring index weight matrix, and calculate the similarity between samples and the neighborhood radius in combination with the monitoring index weight matrix to obtain a dimensionality-reduced feature data set;
[0007] Input the dimensionality-reduced feature data set into a three-layer neural network structure, and perform parallel training on each monitoring point to obtain a distributed prediction model and a training output result;
[0008] Establish an n-level state space according to the training output result, and obtain a hierarchical early warning threshold set through the Markov decision algorithm;
[0009] Input the real-time monitored feature data into the distributed prediction model for abnormal state prediction to obtain the abnormal state prediction value, and compare the abnormal state prediction value with the hierarchical early warning threshold set to output the early warning level signal.
[0010] The present invention also provides an ocean AI analysis and ocean disaster prediction system based on multi-source heterogeneous data, including:
[0011] A calculation module for constructing a data matrix and calculating the information amount of ocean monitoring data to obtain a monitoring index weight matrix, and combining the monitoring index weight matrix to calculate the similarity between samples and the neighborhood radius to obtain a dimensionality-reduced feature data set;
[0012] A parallel training module for inputting the dimensionality-reduced feature data set into a three-layer neural network structure to perform parallel training on each monitoring point to obtain a distributed prediction model and a training output result;
[0013] A decision-making module for establishing an n-level state space according to the training output result and obtaining a hierarchical early warning threshold set through the Markov decision algorithm;
[0014] An output module for inputting the real-time monitored feature data into the distributed prediction model for abnormal state prediction to obtain the abnormal state prediction value, and comparing the abnormal state prediction value with the hierarchical early warning threshold set to output the early warning level signal.
[0015] In summary, the technical solution provided by the present invention effectively reduces the data dimension, retains the key feature information, and reduces the consumption of computing resources through the feature extraction method combining the Critic weighting method and the neighborhood rough set; uses a distributed neural network structure for parallel training, reduces the model training time, and improves the system's ability to process large-scale data; the multi-level early warning mechanism based on the Markov decision process enables the system to dynamically adjust the early warning strategy according to different abnormal degrees, reducing the false alarm rate; by updating the state transition probability and early warning threshold in real time, the adaptability of the system to the dynamic changes of the ocean environment is enhanced; the hierarchical early warning threshold structure is adopted to make the early warning decision more targeted, facilitating the timely adoption of corresponding disaster prevention and mitigation measures, thereby improving the accuracy of ocean AI analysis and ocean disaster prediction. Description of the Drawings
[0016] Figure 1 is a schematic diagram of the steps of an ocean AI analysis and ocean disaster prediction method based on multi-source heterogeneous data in an embodiment of the present invention;
[0017] Figure 2 is a block diagram of the structure of an ocean AI analysis and ocean disaster prediction system based on multi-source heterogeneous data in an embodiment of the present invention.
[0018] The realization, functional features, and advantages of the present invention will be further described in conjunction with embodiments with reference to the accompanying drawings. Detailed Embodiment
[0019] In order to make the objectives, technical solutions, and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0020] Referring to Figure 1 , this embodiment provides a method for ocean AI analysis and ocean disaster prediction based on multi-source heterogeneous data, including the following steps:
[0021] S1. Construct a data matrix and calculate the information volume for ocean monitoring data to obtain a monitoring index weight matrix, and calculate the similarity between samples and the neighborhood radius in combination with the monitoring index weight matrix to obtain a dimensionality-reduced feature dataset;
[0022] Among them, by collecting ocean monitoring data, including key indicators such as temperature, salinity, dissolved oxygen, pH value, and chlorophyll concentration, and sorting and arranging these data in the order of collection time to form an initial data matrix, where each row represents the sampling data at different time points and each column corresponds to a specific monitoring index. To avoid deviations in subsequent analysis caused by different measurement units of different monitoring indicators, the initial data is standardized to map all data to a unified numerical range to obtain a standardized data matrix. Calculate the data volatility of each monitoring index. By analyzing the fluctuation range of the monitoring data, evaluate the dispersion degree of each index, that is, their changes in time and space. The size of the dispersion degree is related to the sensitivity of the index to the overall system state. Analyze the correlation between monitoring indicators and evaluate the dependence relationship between different indicators. By comparing the data change trends of every two monitoring indicators, calculate the degree of correlation between them. Combine the analysis results of the volatility and correlation of monitoring indicators to calculate the comprehensive information volume of each monitoring index. The higher the information volume of an index, the more significant its volatility and the more independent it is from other indicators. This information volume can quantify the importance of each index in the overall system. Normalize the comprehensive information volume of each monitoring index to form a weight matrix. Each element of the weight matrix represents the weight ratio of the corresponding monitoring index in the overall analysis. The weight distribution can highlight the most important indicators and weaken the secondary factors with less impact on the system state. Calculate the similarity between samples according to the weight matrix. Similarity analysis evaluates the closeness of their comprehensive states by comparing the monitoring data at different time points or spatial positions. Based on the similarity between samples, apply a clustering algorithm to determine the neighborhood radius of the data distribution, realize the dimensionality reduction of the original data features, and generate a dimensionality-reduced feature dataset.
[0023] Perform weighted processing on each weight value in the monitoring index weight matrix and the data of the corresponding monitoring index. By multiplying the original data of each monitoring index with its weight in the weight matrix item by item, a weighted feature vector is generated. Calculate the Euclidean distance between pairs of sample data in the weighted feature vector to evaluate the similarity between samples, and generate a sample distance matrix, where each element represents the distance between two sets of samples in the feature space. The sample distance matrix reflects the degree of difference between different samples. Perform normalization processing on the initial similarity data to standardize all distance values to a fixed range, obtaining standardized similarity values. According to the data distribution characteristics of the standardized similarity values, set the neighborhood radius parameter. To ensure that the selection of the neighborhood radius can reflect the true distribution characteristics of the data, the interquartile range method is used to determine the optimal neighborhood radius value. The interquartile range method extracts the range of data variation by statistically analyzing the median and the upper and lower quartiles of the data distribution, thereby setting a reasonable neighborhood radius value that can adapt to the data density. This method is suitable for processing scenarios where the sample distribution has a certain degree of non-uniformity and helps to improve the accuracy of neighborhood relationships. Based on the standardized similarity values and the optimal neighborhood radius value, construct the neighborhood relationships between samples. If the similarity value between two samples is less than or equal to the neighborhood radius value, then these two samples are considered to belong to the same neighborhood, thus establishing a neighborhood relationship. Through this step, a neighborhood relationship set is generated, which contains all sample pairs that meet the neighborhood conditions. The neighborhood relationship set describes the distribution pattern of data samples in the feature space and their local correlation. Convert the neighborhood relationship set into a binary matrix representation to construct a neighborhood decision matrix. The neighborhood decision matrix is a symmetric matrix, where each element only takes two values. If there is a neighborhood relationship between two samples, the value of the corresponding element in the matrix is 1; otherwise, it is 0. Through the neighborhood decision matrix, the neighborhood relationships between samples are comprehensively and compactly described in matrix form. Based on the neighborhood decision matrix, calculate the dependence degree of the conditional attribute set on the decision attribute set to generate a dimensionality-reduced feature data set. The calculation process of the dependence degree reflects the importance of the conditional attribute set to the decision attribute set. By screening features with high dependence degrees, redundant information is effectively reduced, and the expression ability and processing efficiency of the data are improved.
[0024] Calculate the upper approximation set and the lower approximation set between samples according to the neighborhood decision matrix. The partitioning of these two sets is based on the conditional attributes, representing the ranges where samples may belong to a certain class and definitely belong to a certain class respectively. Through the upper approximation set and the lower approximation set, further partition the conditional attribute space to form a partition structure of the conditional attributes, revealing the similarities and differences between samples. Calculate the positive region of the conditional attribute space. By identifying which samples can be clearly classified into the decision attribute space, obtain the dependence degree value of the conditional attribute set C on the decision attribute set D. The dependence degree value characterizes the contribution degree of the conditional attribute set to the target decision classification. The higher the value, the stronger the explanatory ability of the conditional attributes for the decision attributes. Traverse the subset S of the conditional attribute set C according to the forward search strategy. Calculate the dependence degree value of each feature subset on the decision attribute set separately, obtain the contribution degree of each feature to the decision classification, and generate a feature importance sequence. The feature importance sequence is sorted according to the contribution of the features to the decision. Based on the feature importance sequence, set a dependence degree gain threshold interval to screen out the optimal feature subset. The interval setting of the dependence degree gain threshold is based on the distribution of the feature importance sequence, and equidistant sampling is performed within the interval to generate a candidate threshold set. Each threshold in the candidate threshold set represents a potential screening criterion, and its specific role is to determine which features can be retained in the dimensionality-reduced feature dataset. Perform cross-validation on each threshold in the candidate threshold set. By calculating the classification accuracy of the feature subset under different thresholds, evaluate the pros and cons of each threshold. During the cross-validation process, the classification accuracy is used as a measure to judge the impact of feature screening on the model performance. By comparing the classification accuracies corresponding to all thresholds, select the threshold that can maximize the classification accuracy as the optimal threshold. Screen the features according to the optimal threshold. During the screening process, retain the features with feature importance greater than the optimal threshold, and discard other features. Combine the screened features into a dimensionality-reduced feature dataset.
[0025] S2. Input the dimensionality-reduced feature dataset into a three-layer neural network structure, perform parallel training on each monitoring point to obtain a distributed prediction model and training output results;
[0026] Specifically, the dimension-reduced feature dataset is divided into a training dataset and a validation dataset to ensure that the model can evaluate its generalization performance through the validation set during the training process. The training dataset is used to update the model parameters, while the validation dataset is used to detect whether the model is overfitted or under-optimized. After the data division is completed, batch random sampling is performed on the training dataset, and the data is split into multiple small batches to generate neural network training batches. This method helps to improve the training efficiency and avoid biases introduced by data order problems. A three-layer neural network structure including an input layer, a hidden layer, and an output layer is constructed. In this structure, the number of nodes in the input layer is set to the feature dimension of the dimension-reduced feature dataset to ensure that the input data can be completely received. The number of nodes in the hidden layer is set to 2e+1, where e is the feature dimension of the input data. This design enhances the ability to capture complex patterns by increasing the non-linear expression ability of the network. The number of nodes in the output layer is fixed at 1 and is used to predict the value of the target variable. The input layer uses linear transformation to process the input data and maps it to the hidden layer; the hidden layer uses the ReLU activation function for non-linear transformation. This activation function introduces non-linearity and reduces the problem of gradient disappearance; while the output layer uses the Sigmoid activation function to compress the predicted value into the interval [0,1] for probability output or classification tasks. After the network structure is constructed, the connection weights of the three-layer neural network are initialized. The weight initialization uses a random method to avoid the network falling into a symmetric state or gradient disappearance. The Adam optimizer is used to optimize the parameters of the neural network. The Adam optimizer combines momentum and adaptive learning rate adjustment techniques to achieve fast convergence and performs well in dealing with sparse gradient problems. During the training process, the training batch data is input into the neural network, and the backpropagation algorithm is used to calculate the gradients of the loss function with respect to each network parameter. Relying on these gradient information, the weights and bias parameters of the network are updated to gradually approximate the optimal state. To ensure that the model can avoid overfitting, after each training epoch, the validation dataset is input into the updated neural network structure, and the loss value of the validation set is calculated. This process is used to dynamically evaluate the performance of the model on unseen data, providing a basis for triggering the early stopping mechanism. When the validation set loss value fails to further decrease within consecutive G training epochs, it is considered that the model has reached a performance bottleneck, and at this time, the early stopping mechanism is automatically triggered to stop training to avoid overfitting problems. Through this process, an optimized distributed prediction model and corresponding training output results are obtained.
[0027] S3. Establish an n-level state space according to the training output results, and obtain a hierarchical early warning threshold set through the Markov decision algorithm;
[0028] It should be noted that according to the training output results of the distributed prediction model, the degree of abnormality of the ocean state is classified into levels. The classification basis is the range of abnormal values output by the model or the actual severity of abnormal events. For example, the degree of abnormality is divided into n levels such as low-level abnormality, medium-level abnormality, and high-level abnormality. Each level corresponds to a clear state identifier, and an n-level state space is established to represent the ocean state under different abnormal levels. After the state space is established, the changes between different states are dynamically analyzed, and the transfer situation between states is recorded by counting the frequency of state changes at adjacent times. At each time point, the change between the current state and the next state is recorded as a state transfer. By accumulating these transfer times, the total number of transfers of each state to other states is obtained. Based on the transfer times, the transfer ratio of each state is calculated to form the transfer probability data between states. The transfer probability data reflects the possibility of the system transferring from one state to other states. The transfer probability data is organized into a matrix form to generate a state transfer probability matrix. The rows of this matrix represent the current state, the columns represent the possible states at the next moment, and each element in the matrix represents the probability value of the corresponding state transfer. This matrix structure shows the dynamic evolution law of the system between different states. A reward mechanism is set. By analyzing historical data or actual requirements, reward scores are assigned for the warning performance of different states. For the states that can accurately and timely trigger warnings, positive rewards are given to encourage this behavior; while for the states that generate lagged warnings or false alarms, negative rewards are given to reflect their costs. These reward scores constitute the state reward function, which guides the Markov decision algorithm to optimize the response behavior of the warning system. The state transfer probability matrix and the state reward function are input into the Markov decision algorithm, and the decision values of each state are calculated through multiple rounds of iteration. The Markov decision algorithm gradually optimizes the value of each state by seeking a balance between state transfer and reward scores until these values converge and stabilize. The converged state values reflect the relative importance and optimal behavior choices of each state in the overall warning decision. According to the converged optimal decision values, corresponding warning thresholds are set for each state. The setting of the warning thresholds is based on the degree of abnormality of the state and is arranged in ascending order to ensure that the warning system can gradually upgrade the response intensity. The hierarchical warning threshold set effectively guides the warning system to trigger corresponding response levels in the face of different degrees of abnormality through a clear hierarchical structure. The finally generated hierarchical warning threshold set covers all levels from low-level abnormality to high-level abnormality and takes into account the accuracy and response timeliness of warnings.
[0029] S4, input the real-time monitored characteristic data into the distributed prediction model for abnormal state prediction to obtain the abnormal state prediction value, and compare the abnormal state prediction value with the hierarchical warning threshold set to output the warning level signal.
[0030] Specifically, feature extraction is performed on the real-time monitored feature data, and it is dimensionally reduced according to the same feature dimensions as the dimensionally reduced feature dataset to obtain a real-time feature vector. The real-time feature vector is input into the distributed prediction model. The distributed prediction model uses the parameters optimized during the training process through its three-layer neural network structure to perform forward propagation calculations on the input data and generate an abnormal state prediction value. This prediction value is a quantitative assessment of the current ocean monitoring state, indicating the degree of abnormality of the current environment, and its range is normalized to the interval [0,1]. The abnormal state prediction value is compared with the thresholds of each level defined in the hierarchical early warning threshold set, and the abnormal level to which the current state belongs is determined by judging the threshold interval range into which the prediction value falls. The classification of abnormal levels includes low-level abnormality, medium-level abnormality, high-level abnormality, etc. from low to high, reflecting the severity of the current state. Through this step, an abnormal state prediction result is generated. The state transition probability matrix is updated according to the abnormal state prediction result. By recording the number of transitions from the current state to the next state and statistically analyzing these transition data, the state transition data is dynamically updated. This update can reflect the latest real-time state changes, enabling the state transition probability matrix to adapt to new monitoring data in real time, thereby more accurately describing the dynamic behavior of the system. The updated state transition data is input into the Markov decision process. The Markov decision algorithm recalculates the state value function of each state based on the new state transition probability matrix and the previously defined state reward function. Through multiple rounds of iterative calculations, the state value function finally converges to a stable optimal state value. Based on the newly calculated state value, the hierarchical early warning threshold set is dynamically adjusted to adapt to the changes in the current environment, and an updated hierarchical early warning threshold set is obtained. An early warning level signal corresponding to the abnormal level of the abnormal state prediction result is generated. For example, a low-level abnormality triggers a green signal to remind the system that it is in a safe state; a medium-level abnormality triggers a yellow signal to indicate potential risks; a high-level abnormality triggers a red signal to warn that there are major risks that require immediate response.
[0031] In one example, a data matrix is constructed for the ocean monitoring data and the information amount is calculated to obtain a monitoring index weight matrix. Combining the monitoring index weight matrix, the similarity between samples and the neighborhood radius are calculated to obtain a dimensionally reduced feature dataset, including:
[0032] Arrange the monitoring data of ocean temperature, salinity, dissolved oxygen, pH value, and chlorophyll in the order of collection time, and establish an initial data matrix with h rows of sample numbers and f columns of feature numbers for the monitoring data;
[0033] Perform maximum-minimum normalization operations on the data in the initial data matrix to obtain a normalized data matrix, and calculate the standard deviation according to the monitoring index columns of the normalized data matrix to analyze the dispersion degree of the data samples of each monitoring index, and obtain the standard deviation data of f monitoring indexes;
[0034] Pair the standardized data matrix in pairs according to the monitoring indicators for correlation analysis, calculate the Pearson correlation coefficient between the paired monitoring indicators, and obtain an f-order correlation coefficient matrix;
[0035] Calculate the information content of each monitoring indicator based on the standard deviation data and the correlation coefficient matrix, multiply the standard deviation of each monitoring indicator by the linear combination value of the correlation coefficient of this monitoring indicator relative to other monitoring indicators, and obtain the monitoring indicator information content data;
[0036] Normalize the monitoring indicator information content data, divide the information content of each monitoring indicator by the sum of the information content of all monitoring indicators, and obtain the monitoring indicator weight matrix;
[0037] Calculate the similarity between samples and the neighborhood radius in combination with the monitoring indicator weight matrix to generate a dimensionality-reduced feature data set.
[0038] In this example, the monitoring data of ocean temperature, salinity, dissolved oxygen, pH value, and chlorophyll are arranged in chronological order of collection. These data are constructed into an initial data matrix. Assume that the number of monitoring samples is , and the number of monitoring indicators is , then the dimension of the initial data matrix is . The element in the matrix represents the observed value of the th sample under the th monitoring indicator. To eliminate the influence of the numerical ranges of different monitoring indicators on subsequent analysis, perform a maximum-minimum standardization operation on the initial data matrix . The standardization formula is:
[0039] ;
[0040] where, is the original data value, and are respectively the minimum and maximum values in the th column (i.e., the monitoring indicator), is the standardized data value. Map all data to the interval [0,1], and the standardized matrix is denoted as . Calculate the standard deviation of each monitoring indicator in the standardized data matrix . The standard deviation is an important indicator to measure the degree of data dispersion, and its calculation formula is:
[0041] ;
[0042] where, is the standard deviation of the th monitoring indicator, is the mean of the th column, representing the average value of this monitoring indicator. is the number of samples. Through this step, the standard deviation data of monitoring indicators are obtained, reflecting the volatility of each indicator. Perform correlation analysis between monitoring indicators. By pairing the monitoring indicators in two by two and calculating the Pearson correlation coefficient between them. The calculation formula of the correlation coefficient is:
[0043] ;
[0044] where is the correlation coefficient between the th and the th monitoring indicators, and the value range is [-1, 1]. The closer it is to 1, the stronger the positive correlation, and the closer it is to -1, the stronger the negative correlation. By calculating all possible pairs, a correlation coefficient matrix is constructed, where . Based on the standard deviation data and the correlation coefficient matrix , calculate the information content of each monitoring indicator. The calculation formula of the information content is:
[0045] ;
[0046] where represents the information content of the th monitoring indicator, is the standard deviation of this indicator, is the correlation between this indicator and other monitoring indicators, and represents the average correlation between this indicator and all other indicators. Through this formula, the information content comprehensively considers the volatility and independence of the indicator. In order to make the information content comparable, normalize the information content data to obtain the monitoring indicator weight matrix . The normalization formula is:
[0047] ;
[0048] where is the weight of the th monitoring indicator, and the normalized weight matrix reflects the relative importance of each indicator to the system state. Using the weight matrix , calculate the similarity between samples. The similarity between samples is represented by the weighted Euclidean distance, and its formula is:
[0049] ;
[0050] Among them, is the weighted distance between a sample and a sample. Based on these distances, the neighborhood radius is calculated through a density clustering algorithm (such as DBSCAN), and a dimensionality-reduced feature dataset is generated.
[0051] In one example, the similarity between samples and the neighborhood radius are calculated in combination with the monitoring index weight matrix to generate a dimensionality-reduced feature dataset, including:
[0052] The weight values in the monitoring index weight matrix are weighted with the corresponding monitoring index data to obtain weighted feature vectors;
[0053] The Euclidean distances between the sample data in the weighted feature vectors are calculated pairwise to construct a sample distance matrix, obtaining initial similarity data, and the initial similarity data is normalized to obtain standardized similarity values;
[0054] Based on the data distribution characteristics of the standardized similarity values, the neighborhood radius parameter is set, and the quartile range method is used to determine the optimal neighborhood radius value;
[0055] According to the standardized similarity values and the optimal neighborhood radius value, a sample neighborhood relationship is constructed. If the similarity value between samples is less than or equal to the neighborhood radius value, a neighborhood relationship is established to obtain a neighborhood relationship set;
[0056] The neighborhood relationship set is converted into a binary matrix representation to construct a neighborhood decision matrix representing the neighborhood relationship between samples;
[0057] Based on the neighborhood decision matrix, the dependence degree of the conditional attribute set on the decision attribute set is calculated to obtain a dimensionality-reduced feature dataset.
[0058] In this example, the weight values in the monitoring index weight matrix are weighted with the corresponding monitoring index data to obtain weighted feature vectors. Let the monitoring index weight matrix be , where represents the weight of the th monitoring index, and . Assume that the standardized monitoring data matrix is , where represents the value of the th sample under the th monitoring index, and the matrix dimension is , where is the number of samples, is the number of monitoring indexes. The calculation formula for the weighted feature vector is:
[0059] ;
[0060] Among them, is the weighted feature vector value of the th sample, reflecting the comprehensive characteristics of the sample on all monitoring indicators. After calculating all samples, a weighted feature vector set is obtained. Based on the weighted feature vector set , the Euclidean distance between pairwise sample data is calculated to construct a sample distance matrix. The Euclidean distance formula between sample and sample is:
[0061] ;
[0062] where represents the weighted Euclidean distance between sample and sample , reflecting the similarity or difference between the two in the feature space. By calculating the distances for all sample pairs, a symmetric sample distance matrix is constructed, where the matrix dimension is , and each element represents the corresponding inter-sample distance. The sample distance matrix is normalized to ensure that the distance values are within a unified range for subsequent similarity calculations. The normalization formula is:
[0063] ;
[0064] where is the normalized similarity value, with a numerical range of [0,1]. The normalized matrix represents the standardized similarity value matrix, reflecting the similarity relationship between samples. Based on the distribution characteristics of the standardized similarity value matrix, a neighborhood radius parameter is set to determine the neighborhood relationship between samples. The interquartile range method is used to determine the optimal neighborhood radius value. Calculate the upper and lower quartiles and of the similarity values, as well as the interquartile range :
[0065] ;
[0066] According to the interquartile range method, the optimal neighborhood radius value is set to:
[0067] ;
[0068] where represents the neighborhood radius value, used to define the neighborhood relationship between samples. According to the optimal neighborhood radius value and the standardized similarity value , a neighborhood relationship set between samples is constructed. If two samples and similarity value is less than or equal to , it is considered that there is a neighborhood relationship between them. The neighborhood relationship set is represented as:
[0069] ;
[0070] Convert the neighborhood relationship set into a binary matrix , where represents that there is a neighborhood relationship between sample and , and represents the absence of a neighborhood relationship. The constructed binary matrix is the neighborhood decision matrix. After obtaining the neighborhood decision matrix, calculate the dependence degree of the conditional attribute set on the decision attribute set. Let the conditional attribute set be , and the decision attribute set be . The definition of the dependence degree is:
[0071] ;
[0072] where, represents the dependence degree of the conditional attribute set on the decision attribute set, is the number of positive domain samples of the decision attribute on the conditional attribute, is the total number of samples. The dependence degree value reflects the importance of the conditional attribute set for classifying the decision attribute. Through the above steps, a dimensionality-reduced feature dataset is generated.
[0073] In an example, based on the neighborhood decision matrix, calculate the dependence degree of the conditional attribute set on the decision attribute set to obtain a dimensionality-reduced feature dataset, including:
[0074] Calculate the upper approximation set and lower approximation set between samples according to the neighborhood decision matrix, obtain the partition of the conditional attribute space, and perform a positive domain calculation on the partition of the conditional attribute space to obtain the dependence degree value of the conditional attribute set C on the decision attribute set D;
[0075] Traverse the subset S of the conditional attribute set C according to the forward search strategy, calculate the dependence degree value of each feature subset on the decision attribute set, and obtain the feature importance sequence;
[0076] Based on the feature importance sequence, set the dependence degree gain threshold interval, and perform equidistant sampling within the interval range to obtain the candidate threshold set;
[0077] Perform cross-validation on each threshold in the candidate threshold set, calculate the classification accuracy of the feature subsets under different thresholds, obtain the optimal threshold, and screen the features according to the optimal threshold. The features with importance greater than the threshold are composed into a dimensionality-reduced feature dataset.
[0078] In this example, the upper approximation set and the lower approximation set between samples are calculated based on the neighborhood decision matrix, so as to partition the conditional attribute space and complete the positive region calculation. Let the neighborhood decision matrix be , where represents whether there is a neighborhood relationship between sample and sample (1 means yes, 0 means no). Assume the sample set is , and each sample has a decision attribute and the definition of the lower approximation set is as follows:
[0079] ;
[0080] ;
[0081] Among them, represents the sample set with the same decision attribute as sample . Through the partition of the upper approximation and the lower approximation, the conditional attribute space is structured. After completing the upper approximation and lower approximation calculations, the size of the positive region set determines the dependence degree of the conditional attribute set on the decision attribute set . The dependence degree formula is defined as:
[0082] ;
[0083] Among them, is the number of samples in the positive region set, is the total number of samples. The larger the dependence degree value , the higher the importance of the conditional attribute set for decision attribute classification. Traverse all subsets of the conditional attribute set according to the forward search strategy, and calculate the dependence degree value of each subset on the decision attribute set. Assume , and gradually select subsets through forward search, and calculate their dependence degree values in turn:
[0084] ;
[0085] Among them, is the subset The size of the positive domain set. The dependence degree values of all subsets form a feature importance sequence, and each value in the sequence reflects the contribution of the corresponding feature subset to the classification of the decision attribute. After obtaining the feature importance sequence, set the dependence degree gain threshold interval to select appropriate features from the sequence. Let the maximum value of the feature importance sequence be , and the minimum value be . Then the threshold interval is defined as:
[0086] ;
[0087] Generate a candidate threshold set by equidistant sampling within this interval range . Each threshold represents a feature screening criterion, and select the feature subset with importance greater than for further verification. To evaluate the effect of each candidate threshold, use the cross-validation method to calculate the classification accuracy of the feature subset under different thresholds. The cross-validation is divided into the following steps: divide the data into a training set and a validation set, screen the feature subset based on the current threshold , then use the training set to train the model, and use the validation set to calculate the classification accuracy. Let the classification accuracy be . By comparing the values of all candidate thresholds, select the threshold that maximizes the classification accuracy as the optimal threshold. According to the optimal threshold , screen out the feature subset with importance greater than , and use it as the dimensionality-reduced feature data set.
[0088] After obtaining the dimensionality-reduced feature dataset by screening features according to the optimal threshold and before dividing the dimensionality-reduced feature dataset into a training dataset and a validation dataset, the following steps are also included: Input the dimensionality-reduced feature dataset into a principal component analysis model, calculate the feature covariance matrix and the feature vector matrix, perform a linear transformation on the dimensionality-reduced feature dataset to obtain the initial projection space coordinates; Calculate the k-nearest neighbor point sets for each data point in the initial projection space coordinates, construct a local weight matrix based on the Euclidean distance between the data points and their neighbor points, calculate the similarity values between data point pairs using a Gaussian kernel function to obtain the local structure feature matrix; Calculate the within-class scatter and between-class scatter for the initial projection space coordinates, minimize the projection distances of within-class data points and maximize the projection distances of between-class data points, construct a global scatter matrix to obtain the global structure feature matrix; Construct a double-constraint objective function based on the local structure feature matrix and the global structure feature matrix, set the local structure preservation weight parameter α and the global structure preservation weight parameter β, solve the objective function through an alternating optimization algorithm to obtain the optimal projection direction; Calculate the importance scores of the projection coordinates according to the optimal projection direction, assign corresponding reweighting coefficients to each dimension of the initial projection space coordinates, set the weight coefficients of important coordinates to values greater than 1, and set the weight coefficients of redundant coordinates to values less than 1 to obtain the coordinate weighting coefficient matrix; Perform a matrix multiplication operation on the coordinate weighting coefficient matrix and the initial projection space coordinates to obtain the reweighted projection subspace; Construct a similarity graph for the reweighted projection subspace, calculate the eigenvalues and eigenvectors of the graph Laplacian matrix, and use the K-means algorithm to cluster the eigenvectors to obtain the subspace clustering result; Use the subspace clustering result as the new feature dimension, perform feature concatenation with the dimensionality-reduced feature dataset, and normalize the concatenated feature matrix to obtain the enhanced feature dataset, which is used for subsequent neural network training.
[0089] In one example, input the dimensionality-reduced feature dataset into a three-layer neural network structure, and perform parallel training on each monitoring point to obtain a distributed prediction model and training output results, including:
[0090] Divide the dimensionality-reduced feature dataset into a training dataset and a validation dataset, perform batch random sampling on the training dataset to obtain neural network training batches;
[0091] Construct a three-layer neural network structure, which includes an input layer, a hidden layer, and an output layer. The number of nodes in the input layer is equal to the feature dimension of the dimensionality-reduced feature dataset, the number of nodes in the hidden layer is 2e+1, and the number of nodes in the output layer is 1; The input layer uses a linear transformation to process the input data, the hidden layer uses a ReLU activation function for nonlinear transformation, and the output layer uses a Sigmoid activation function to map the output value to the interval [0,1] to obtain the prediction result;
[0092] Initialize the connection weights in the three-layer neural network structure, use the Adam optimizer for parameter optimization, and input the neural network training batch data into the three-layer neural network structure. Calculate the gradients using the backpropagation algorithm and update the network parameters;
[0093] After each training epoch, input the validation dataset into the updated neural network for validation. When the validation set loss value has not decreased for G consecutive epochs, trigger the early stopping mechanism to obtain the distributed prediction model and the training output results.
[0094] In this example, the dimensionality-reduced feature dataset is divided into a training dataset and a validation dataset to ensure that the model can optimize the parameters using the training data and monitor the generalization performance of the model using the validation data. Assume the dimensionality-reduced feature dataset is , where represents the features of the th sample, is the feature dimension. The corresponding target label is , where represents the target value of the th sample. Divide the dataset into a training set and a validation set. Let the proportion of the training set be , then the size of the training set is , and the size of the validation set is . After division, the training set is , and the validation set is . Then, perform batch random sampling on the training dataset, divide the training dataset into multiple small batches to improve the training efficiency and stabilize the gradient calculation. Assume the batch size is , then the training data is divided into batches, and each batch is , where , . Construct a three-layer neural network structure, including an input layer, a hidden layer, and an output layer. The number of nodes in the input layer is equal to the feature dimension , the number of nodes in the hidden layer is set to to increase the expressive power of the network, and the number of nodes in the output layer is 1 for predicting the target value. The forward propagation process of the neural network includes three stages. In the input layer, perform a linear transformation on the input data , and the calculation formula is:
[0095] ;
[0096] where, is the weight matrix from the input layer to the hidden layer, is the bias vector, is the result of the linear transformation. In the hidden layer, for The ReLU activation function is used to achieve non - linear transformation, and its calculation formula is:
[0097] ;
[0098] where, is the output of the hidden layer. Then, is passed to the output layer for linear transformation and Sigmoid activation processing, and the calculation formula is:
[0099] ;
[0100] ;
[0101] where, is the weight matrix from the hidden layer to the output layer, is the bias of the output layer, is the prediction result. After defining the network structure, all weights 、 and biases 、 are initialized, using uniform distribution or Xavier initialization method to ensure that the parameter values are within a reasonable range. During the training process, the Adam optimizer is used for parameter optimization. This optimizer combines the advantages of momentum and adaptive learning rate adjustment to improve the convergence speed. The loss function uses cross - entropy loss, which is defined as:
[0102] ;
[0103] where, is the actual value, is the predicted value, is the batch size. The gradient is calculated through the backpropagation algorithm, and the weights and biases are updated using the Adam optimizer. The gradient update formula is:
[0104] ;
[0105] where, is the learning rate. After each training epoch ends, the validation dataset is input into the neural network for validation, and the validation loss is calculated. If the validation loss does not decrease within consecutive epochs, the early stopping mechanism is triggered to stop training to prevent overfitting. Finally, a distributed prediction model and training output results are obtained.
[0106] In an example, an n - level state space is established according to the training output results, and a hierarchical early warning threshold set is obtained through the Markov decision algorithm, including:
[0107] The training output results of the distributed prediction model are classified into levels according to the degree of ocean anomalies, divided into n levels such as low - degree anomaly, medium - degree anomaly, and high - degree anomaly. Corresponding status identifiers are assigned to each level to obtain an n - level state space;
[0108] Statistical analysis is carried out on the state changes of adjacent moments in the n - level state space, the number of state transitions is recorded, and proportional calculation is performed according to the total number of transitions to obtain the state - to - state transition probability data;
[0109] The state - to - state transition probability data is constructed in matrix form, where the rows of the matrix represent the current state, the columns represent the next - moment state, and the values represent the corresponding transition probabilities, to obtain the state transition probability matrix;
[0110] Based on the accuracy of ocean warning decisions and the timeliness of warning responses, reward scores are set. Positive rewards are given to the states with accurate and timely warnings, and negative rewards are given to the states with lagged or false warnings to obtain the state reward function;
[0111] The state transition probability matrix and the state reward function are input into the Markov decision algorithm, and multi - round iterative calculations are performed on each state until the state values converge and stabilize to obtain the optimal decision values of each state;
[0112] According to the optimal decision values of each state, corresponding warning thresholds are set, and the warning thresholds are sorted in ascending order of the degree of anomaly to obtain a hierarchical warning threshold set.
[0113] In this example, the training output results of the distributed prediction model are processed. The prediction results are divided into multiple levels according to the degree of ocean anomalies to construct an n - level state space. Let the output result of the prediction model be , where represents the degree of anomaly at the th moment. Based on the pre - defined threshold , is divided into levels, where each level corresponds to a status identifier. The division rule of the status identifier is:
[0114] ;
[0115] where, is the status identifier at the th moment, indicating that the current degree of anomaly belongs to levels such as low - degree anomaly (status 1), medium - degree anomaly (status 2), or high - degree anomaly (status ), etc. Through state mapping of the data at all moments, The zero-level state space. After constructing the state space, statistical analysis is performed on the state changes at adjacent moments to record the number of state transitions. Let the state sequence be , and count the occurrence frequency of each pair of adjacent states , where , and represent the identifiers of the current state and the next moment state respectively. By accumulating all the transition times, the frequency matrix of state transitions is obtained, where represents the total number of times transferred from state to state . Calculate the transition probability according to the total number of transitions of each state. Let represent the probability of transferring from state to state , then there is:
[0116] ;
[0117] where, is the total number of transitions of state . Through normalization calculation, construct the state transition probability matrix , the matrix dimension is , each row represents the current state, each column represents the possible states at the next moment, and the matrix element represents the corresponding transition probability. Combining the actual needs of marine early warning, set a reward function to quantify the accuracy and response timeliness of early warning. The reward function is a two-dimensional matrix, where represents the reward value obtained by transferring from state to state . Suppose that the transfer of accurate and timely early warning (such as maintaining low-level anomaly from low-level anomaly) obtains a positive reward , while false alarm (such as transferring from low-level anomaly to high-level anomaly) or delayed early warning (such as transferring from high-level anomaly to low-level anomaly) obtains a negative reward . Input the state transition probability matrix and the reward function matrix into the Markov decision algorithm to calculate the optimal decision value of each state. The Markov decision algorithm calculates the state value function through iteration, which represents the optimal value at state . The iterative formula of the state value function is:
[0118] ;
[0119] where, is the discount factor, which is used to balance short-term and long-term rewards. Through multiple rounds of iterative calculations, the state value function eventually converges to obtain the optimal decision value for each state. Based on the optimal decision values of each state , the corresponding warning thresholds are set . The principle of threshold setting is to adjust according to the order of state values from low to high, ensuring that the threshold for low-abnormality states is the lowest and the threshold for high-abnormality states is the highest. These thresholds form a hierarchical warning threshold set .
[0120] In an example, the real-time monitored feature data is input into a distributed prediction model for abnormal state prediction to obtain an abnormal state prediction value, and the abnormal state prediction value is compared with the hierarchical warning threshold set to output a warning level signal, including:
[0121] Feature extraction is performed on the real-time monitored feature data, and data dimensionality reduction processing is carried out according to the same feature dimensions as the dimensionality-reduced feature dataset to obtain a real-time feature vector;
[0122] The real-time feature vector is input into the distributed prediction model, and forward propagation calculation is performed using the trained parameters in the three-layer neural network structure to obtain an abnormal state prediction value;
[0123] The size of the abnormal state prediction value is compared with the thresholds of each level in the hierarchical warning threshold set, and the abnormal level to which the current state belongs is determined based on the threshold interval range to obtain an abnormal state prediction result;
[0124] The state transition probability matrix is updated according to the abnormal state prediction result, and the transition situation from the current state to the next predicted state is recorded to obtain updated state transition data;
[0125] The updated state transition data is input into the Markov decision process, the state value function is recalculated, and the warning thresholds are dynamically adjusted to obtain an updated hierarchical warning threshold set;
[0126] A warning level signal of the corresponding level is generated according to the abnormal level corresponding to the abnormal state prediction result.
[0127] In this example, feature extraction is performed on the real-time collected monitored feature data. Let the original monitored data matrix be , and its dimension is , where is the number of real-time collected samples, and is the original feature dimension. In order to maintain the same feature dimension as the dimensionality-reduced feature dataset , is projected into the dimensionality-reduced feature space. Let the dimensionality reduction transformation matrix be , then the real-time feature vector The calculation formula is:
[0128] ;
[0129] wherein, is a transformation matrix obtained by a dimensionality reduction algorithm (such as principal component analysis or feature selection), is the feature vector after dimensionality reduction of the real-time monitoring data. The real-time feature vector is input into the already trained distributed prediction model. This model consists of a three-layer neural network structure, including an input layer, a hidden layer, and an output layer, and performs forward propagation calculation using the parameters optimized in the training phase. Let the real-time feature vector be the feature of the th sample. The calculation process of the three-layer neural network is as follows. A linear transformation is performed in the input layer:
[0130] ;
[0131] wherein, is the weight matrix from the input layer to the hidden layer, is the bias vector, is the input of the hidden layer, is the number of nodes in the hidden layer (usually ). In the hidden layer, the ReLU activation function is applied for non-linear transformation:
[0132] ;
[0133] The output of the hidden layer is passed to the output layer, and the abnormal state prediction value is obtained through linear transformation and the Sigmoid activation function:
[0134] ;
[0135] ;
[0136] wherein, is the weight matrix from the hidden layer to the output layer, is the bias of the output layer, is the abnormal state prediction value of the th sample. The prediction value is compared with the threshold defined in the hierarchical early warning threshold set. By judging the threshold interval where is located, the abnormal level to which the current state belongs is determined:
[0137] ;
[0138] wherein, It is the level identifier of the abnormal state, indicating low-level abnormality (1), medium-level abnormality (2), or high-level abnormality and other levels. According to the abnormal state prediction results obtained from real-time monitoring, update the state transition probability matrix . Let the initial state transition matrix be , where represents the probability of transitioning from state to state . By recording the changes in the real-time state, accumulate the number of transitions into and update the probability matrix by row normalization:
[0139] ;
[0140] The updated state transition matrix reflects the change pattern of the real-time state. Input the updated state transition matrix and the previously set reward function into the Markov decision process. Through multiple rounds of iterative calculation of the state value function , adjust the optimal decision value of each state. The iterative formula of the state value function is:
[0141] ;
[0142] where is the discount factor, used to balance short-term and long-term rewards. Iterate until converges. According to the converged state value function , dynamically adjust the warning threshold to adapt to the changing trend of real-time data. Generate warning signals of corresponding levels according to the level corresponding to the abnormal state prediction results.
[0143] Referring to Figure 2 , this embodiment provides an ocean AI analysis and ocean disaster prediction system based on multi-source heterogeneous data, including:
[0144] Calculation module 1, used to construct a data matrix and calculate the information volume for ocean monitoring data, obtain the monitoring index weight matrix, and calculate the similarity between samples and the neighborhood radius in combination with the monitoring index weight matrix to obtain a dimensionality-reduced feature dataset;
[0145] Parallel training module 2, used to input the dimensionality-reduced feature dataset into a three-layer neural network structure, perform parallel training on each monitoring point, and obtain a distributed prediction model and training output results;
[0146] Decision module 3, used to establish an n-level state space according to the training output results and obtain a hierarchical warning threshold set through the Markov decision algorithm;
[0147] An output module 4 is configured to input the real-time monitored feature data into a distributed prediction model for predicting an abnormal state, obtain an abnormal state prediction value, compare the abnormal state prediction value with a hierarchical warning threshold set, and output a warning level signal.
[0148] In this embodiment, for the specific implementation of each unit in the above system embodiment, please refer to that described in the above method embodiment, and details are not described herein again.
[0149] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above method embodiments. Among them, any reference to a memory, storage, database, or other medium provided in the present invention and used in the embodiments can include non-volatile and / or volatile memories. Non-volatile memories can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memories can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM, etc.
[0150] It should be noted that in this article, the terms "including", "comprising", or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, system, article, or method including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, system, article, or method. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, system, article, or method including that element.
[0151] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.
Claims
1. A marine AI analysis and marine disaster prediction method based on multi-source heterogeneous data, characterized in that: The following steps are involved: Constructing a data matrix and calculating the amount of information on the marine monitoring data to obtain a monitoring indicator weight matrix, and combining the monitoring indicator weight matrix to calculate the similarity between samples and the neighborhood radius to obtain a reduced-dimensional feature data set; The dimension-reduced feature data set is input into a three-layer neural network structure, and each monitoring point is trained in parallel to obtain a distributed prediction model and training output results; According to the training output results, an n-level state space is established, and a hierarchical warning threshold set is obtained through a Markov decision algorithm, specifically including: dividing the training output results of the distributed prediction model into n levels according to the degree of ocean anomaly, wherein the degree of anomaly includes low anomaly, moderate anomaly, and high anomaly, and assigning a corresponding state identifier to each level to obtain an n-level state space; statistically analyzing the state changes at adjacent moments in the n-level state space, recording the number of transitions between states, and calculating the proportion according to the total number of transitions to obtain the state transition probability data; constructing the state transition probability data into a matrix form, wherein the rows in the matrix represent the current state, and the columns represent the next state. The state at a certain moment, the numerical value represents the corresponding transition probability, and the state transition probability matrix is obtained; the reward score is set based on the accuracy of marine warning decision-making and the timeliness of warning response, and a positive reward is given to the state of accurate and timely warning, and a negative reward is given to the state of delayed warning or false alarm, so as to obtain the state reward function; the state transition probability matrix and the state reward function are input into the Markov decision algorithm, and based on the state value function, multiple rounds of iterative calculations are performed on each state until the state value converges and stabilizes, and the optimal decision value of each state is obtained; the corresponding warning threshold is set according to the optimal decision value of each state, and the warning threshold is sorted in order from low to high according to the abnormality degree, so as to obtain a hierarchical warning threshold set; The real-time monitoring feature data is input into the distributed prediction model to predict the abnormal state, and the abnormal state prediction value is obtained, and the abnormal state prediction value is compared with the hierarchical warning threshold set to obtain the abnormal state prediction result; the state transition probability matrix is updated according to the abnormal state prediction result, and the transition from the current state to the next predicted state is recorded to obtain updated state transition data; the updated state transition data is input into the Markov decision algorithm, the state value function is recalculated, and the warning threshold is dynamically adjusted to obtain an updated hierarchical warning threshold set; according to the updated abnormal level corresponding to the abnormal state prediction result, a warning level signal of the corresponding level is finally generated.
2. The method for ocean AI analysis and ocean disaster prediction based on multi-source heterogeneous data according to claim 1 is characterized in that: The data matrix is constructed and the information amount is calculated for the marine monitoring data to obtain a monitoring indicator weight matrix, and the similarity between samples and the neighborhood radius are calculated in combination with the monitoring indicator weight matrix to obtain a reduced dimension feature data set, including: The monitoring data of ocean temperature, salinity, dissolved oxygen, pH value, and chlorophyll are arranged in the order of collection time, and an initial data matrix with h rows of samples and f columns of features is established for the monitoring data; Performing maximum and minimum value normalization operations on the data in the initial data matrix to obtain a standardized data matrix, and performing standard deviation calculations on the standardized data matrix according to the monitoring indicator columns, performing a discrete degree analysis on the data samples of each monitoring indicator, and obtaining standard deviation data of f monitoring indicators; The standardized data matrix is paired with the monitoring indicators for correlation analysis, and the Pearson correlation coefficient between the monitoring indicators is calculated to obtain an f-order correlation coefficient matrix; Calculate the information content of each monitoring indicator based on the standard deviation data and the correlation coefficient matrix, and multiply the standard deviation of each monitoring indicator by the linear combination value of the correlation coefficient of the monitoring indicator relative to other monitoring indicators to obtain the monitoring indicator information content data; Normalizing the monitoring indicator information data, dividing the information of each monitoring indicator by the sum of the information of all monitoring indicators, to obtain a monitoring indicator weight matrix; The monitoring indicator weight matrix is combined to perform similarity calculations between samples and neighborhood radius calculations to generate a reduced-dimensional feature data set.
3. The method for ocean AI analysis and ocean disaster prediction based on multi-source heterogeneous data according to claim 2 is characterized in that: The similarity calculation between samples and the neighborhood radius calculation are performed in combination with the monitoring indicator weight matrix to generate a reduced dimension feature data set, including: Performing weighted calculation on the weight values in the monitoring indicator weight matrix and the data of the corresponding monitoring indicator to obtain a weighted feature vector; Calculating the Euclidean distance between the sample data in the weighted feature vector, constructing a sample distance matrix, obtaining initial similarity data, and normalizing the initial similarity data to obtain a standardized similarity value; The neighborhood radius parameter is set based on the data distribution characteristics of the standardized similarity value, and the optimal neighborhood radius value is determined by using the interquartile range method; Constructing a sample neighborhood relationship according to the standardized similarity value and the optimal neighborhood radius value, and establishing a neighborhood relationship if the similarity value between samples is less than or equal to the neighborhood radius value, thereby obtaining a neighborhood relationship set; The neighborhood relationship set is converted into a binary matrix representation, and a neighborhood decision matrix representing the neighborhood relationship between samples is constructed; The dependency of the condition attribute set on the decision attribute set is calculated based on the neighborhood decision matrix to obtain a reduced-dimensional feature data set.
4. The method for ocean AI analysis and ocean disaster prediction based on multi-source heterogeneous data according to claim 3 is characterized in that: The step of calculating the dependency of the condition attribute set on the decision attribute set based on the neighborhood decision matrix to obtain a reduced-dimensional feature data set includes: According to the neighborhood decision matrix, the upper approximate set and the lower approximate set between samples are calculated to obtain the conditional attribute space division. And perform positive domain calculation on the condition attribute space partition to obtain the dependency value of the condition attribute set C on the decision attribute set D; Traversing the subset S of the condition attribute set C according to the forward search strategy, calculating the dependency value of each feature subset on the decision attribute set, and obtaining a feature importance sequence; Setting a dependency gain threshold interval based on the feature importance sequence, performing equidistant sampling within the interval to obtain a candidate threshold set; Each threshold in the candidate threshold set is cross-validated, the classification accuracy of the feature subset under different thresholds is calculated to obtain the optimal threshold, and the features are screened according to the optimal threshold, and the features with importance greater than the threshold are composed of a reduced dimension feature data set.
5. The method for ocean AI analysis and ocean disaster prediction based on multi-source heterogeneous data according to claim 1 is characterized in that: The reduced dimension feature data set is input into a three-layer neural network structure, and each monitoring point is trained in parallel to obtain a distributed prediction model and training output results, including: Dividing the dimension-reduced feature data set into a training data set and a validation data set, and performing batch random sampling on the training data set to obtain a neural network training batch; Construct a three-layer neural network structure, the three-layer neural network structure includes an input layer, a hidden layer and an output layer, wherein the number of nodes in the input layer is equal to the feature dimension of the dimension reduction feature data set, the number of nodes in the hidden layer is 2e+1, and the number of nodes in the output layer is 1; the input layer uses linear transformation to process input data, the hidden layer uses ReLU activation function for nonlinear transformation, and the output layer uses Sigmoid activation function to map the output value to the [0,1] interval to obtain a prediction result; Initializing the connection weights in the three-layer neural network structure, using the Adam optimizer to optimize the parameters, inputting the neural network training batch data into the three-layer neural network structure, using the back propagation algorithm to calculate the gradient, and updating the network parameters; After each training cycle, the verification data set is input into the updated neural network for verification. When the loss value of the verification set does not decrease for G consecutive cycles, the early stopping mechanism is triggered to obtain a distributed prediction model and training output results.
6. The method for ocean AI analysis and ocean disaster prediction based on multi-source heterogeneous data according to claim 1 is characterized in that: The step of inputting the real-time monitoring feature data into the distributed prediction model to perform abnormal state prediction to obtain an abnormal state prediction value, and comparing the abnormal state prediction value with the hierarchical warning threshold set to obtain an abnormal state prediction result includes: Extracting features from the real-time monitoring feature data, performing data dimension reduction processing according to the same feature dimension as the dimension reduction feature data set, and obtaining a real-time feature vector; The real-time feature vector is input into the distributed prediction model, and the parameters trained in the three-layer neural network structure are used to perform forward propagation calculation to obtain an abnormal state prediction value; The abnormal state prediction value is compared with each level threshold in the hierarchical warning threshold set, and the initial abnormal level to which the current state belongs is determined based on the threshold interval range to obtain the abnormal state prediction result.
7. An ocean AI analysis and ocean disaster prediction system based on multi-source heterogeneous data, characterized in that: For implementing the steps of the method according to any one of claims 1 to 6, the system comprises: A calculation module is used to construct a data matrix and calculate the amount of information of the marine monitoring data to obtain a monitoring indicator weight matrix, and to calculate the similarity between samples and the neighborhood radius in combination with the monitoring indicator weight matrix to obtain a reduced dimension feature data set; A parallel training module is used to input the dimension reduction feature data set into a three-layer neural network structure, perform parallel training on each monitoring point, and obtain a distributed prediction model and training output results; A decision module, used to establish an n-level state space according to the training output results, and obtain a hierarchical warning threshold set through Markov decision making; The output module is used to input the real-time monitoring feature data into the distributed prediction model to predict the abnormal state, obtain the abnormal state prediction value, compare the abnormal state prediction value with the hierarchical warning threshold set, and output the warning level signal.
Citation Information
Patent Citations
Gene selection method and device based on fault tolerance
CN110210552A
Lightweight equipment fault prediction method based on feature extraction and feature fusion
CN117786591A