Incremental frequent item set mining method and caching strategy for unmanned underwater vehicle

Abstract

The invention provides an incremental frequent item set mining method and caching strategy for an unmanned underwater vehicle, and the method comprises the following steps: obtaining the data of a plurality of sensors of the unmanned underwater vehicle, merging the initial frequent item sets of all sensor data, and taking the merged initial frequent item sets as a global candidate item set list; respectively calculating edge distribution of each piece of sensor data and a correlation coefficient between the sensor data, and constructing joint distribution of different pieces of sensor data according to the edge distribution and the correlation coefficient; generating a frequent item set for the newly-added sensor data, calculating the similarity between the newly-added frequent item set and all initial frequent item sets, if the similarity is greater than a set similarity threshold value, merging the newly-added frequent item set into the corresponding initial frequent item set, otherwise, adding the newly-added frequent item set as a new item set into a global candidate item set list; and dynamically adjusting parameters of joint distribution and the support degree of each frequent item set according to the updated global candidate item set list so as to update the global candidate item set list.

Description

Technical Field

[0001] This invention belongs to the field of data mining, specifically relating to an incremental frequent itemset mining method and caching strategy for unmanned underwater vehicles. Background Technology

[0002] In the era of big data, the demand for processing massive amounts of dynamic data is constantly increasing, especially evident in complex system platforms such as unmanned underwater vehicles (UUVs). Various sensors on these platforms generate enormous and continuously growing amounts of data. Traditional data mining methods have revealed many drawbacks when dealing with this type of large-scale dynamic data, such as excessive computational resource consumption and insufficient memory supply. Particularly in the frequent itemset update stage of incremental mining, repeated scanning of the original data is often required, directly leading to extremely low computational efficiency.

[0003] Currently, frequent itemset mining based on multi-scale theory has been introduced into related fields. The advantage of this method lies in its ability to conduct in-depth analysis of data from different business levels and granularities, and to complete incremental updates of frequent itemsets with relatively low computational cost, providing new ideas and approaches for big data processing.

[0004] However, existing technologies still have some shortcomings. Current frequent itemset mining methods struggle to perform incremental frequent itemset mining efficiently and cost-effectively in large-scale dynamic data processing and parallel computing environments. For UUV platforms, with limited data processing resources, traditional incremental frequent itemset mining methods require repeated scanning of the entire dataset when processing large-scale dynamic data, resulting in high computational resource consumption and low efficiency. This problem is further exacerbated by the continuous growth of sensor data in underwater missions. Furthermore, traditional frequent itemset mining methods have high computational complexity when updating frequent itemsets, relying on multiple dataset scans and frequent tree structure adjustments. In real-time data analysis and fault prediction scenarios for UUVs, it is difficult to reduce computational costs and improve computational efficiency while maintaining accuracy. Especially in distributed computing environments, efficient data processing and mining are urgent technical challenges that need to be addressed. Summary of the Invention

[0005] This invention proposes an incremental frequent itemset mining method and caching strategy for unmanned underwater vehicles, which solves the problems of high computational resource consumption and low efficiency of existing frequent itemset mining methods.

[0006] To address the aforementioned technical problems, this invention provides an incremental frequent itemset mining method for unmanned underwater vehicles, comprising the following steps:

[0007] Step S1: Acquire data from multiple sensors of the unmanned underwater vehicle, perform frequent itemset mining on the data from all sensors, generate initial frequent itemsets for each sensor, and merge all initial frequent itemsets as a global candidate itemset list;

[0008] Step S2: Calculate the edge distribution of data from each sensor and the correlation coefficient between the data, and construct the joint distribution of the data based on the edge distribution and correlation coefficient;

[0009] Step S3: Update the global candidate itemset list: Generate frequent itemsets for the new data from the sensors of the unmanned underwater vehicle during navigation, calculate the similarity between the new frequent itemsets and all the initial frequent itemsets, and if the similarity is greater than the set similarity threshold, merge the new frequent itemsets into the corresponding initial frequent itemsets; otherwise, add the new frequent itemsets as a new itemset to the global candidate itemset list.

[0010] Step S4: Dynamically adjust the parameters of the joint distribution based on the updated global candidate itemset list, update the support of frequent itemsets based on the updated joint distribution, and thus update the global candidate itemset list.

[0011] Preferably, the step S1 of generating the initial frequent itemsets for each sensor includes the following steps: performing frequent itemset mining on the data of all sensors to generate candidate frequent itemsets for each sensor, calculating the support of all candidate frequent itemsets, and using the candidate frequent itemsets with support greater than a set threshold as the initial frequent itemsets for each sensor.

[0012] Preferably, the expression for calculating the support is:

[0013] ;

[0014] In the formula, For candidate frequent itemsets Support level; For candidate frequent itemsets The number of times it appears; This represents the total number of transactions.

[0015] Preferably, the expression for calculating the edge distribution of the sensor data in step S2 is:

[0016]

[0017] ;

[0018] ;

[0019] In the formula, For sensor data The marginal distribution; For sensor data The marginal probability density; This refers to the number of samples collected from the sensor data. The bandwidth for kernel density estimation; For kernel functions; Let be the observed value of the i-th sensor data point.

[0020] Preferably, the expression for calculating the correlation coefficient between the data in step S2 is:

[0021] ;

[0022] ;

[0023] ;

[0024] In the above formula, For Copula functions; , These are the edge distribution values ​​of the sensor data x and y; The correlation coefficient between sensor data x and y; for The bivariate standard normal distribution function; It is the inverse function of the standard normal distribution; These are the dependent parameters of the Copula function.

[0025] Preferably, in step S2, when constructing the joint distribution of the data, the parameters of the joint distribution are estimated using historical data from the sensors. The expression for estimating the parameters of the joint distribution is:

[0026] ;

[0027] ;

[0028] In the above formula, Let be the maximum likelihood estimation function; , These are the parameters of the Copula function; This represents the number of samples in the historical sensor data. For Copula functions; , For historical sensor data The marginal distribution.

[0029] Preferably, the expression for constructing the joint distribution of the data based on the marginal distribution and correlation coefficient in step S2 is as follows:

[0030] ;

[0031] In the formula, The joint distribution of sensor data x and y; , These represent the edge distributions of sensor data x and y, respectively. The correlation coefficient between sensor data x and y; , These are the marginal probability densities of sensor data x and y, respectively.

[0032] Preferably, the expression for calculating the similarity between frequent itemsets in step S3 is:

[0033] ;

[0034] In the formula, Jaccard similarity; , Represents sensor data; This represents the frequent itemsets in the sensor data D.

[0035] Preferably, the expression for dynamically adjusting the parameters of the joint distribution based on the updated global candidate set list in step S4 is:

[0036] ;

[0037] In the formula, The correlation coefficient between the updated sensor data x and y; This represents the parameter that maximizes the objective function. The value of ; For newly added sensor data; , These represent the edge distributions of sensor data x and y, respectively. This is the correlation coefficient between sensor data x and y.

[0038] A caching strategy, applicable to the above-mentioned incremental frequent itemset mining method for an unmanned underwater vehicle, includes the following steps:

[0039] Step S1: Use a distributed computing framework to convert sensor data into a resilient distributed dataset RDD, and store all RDDs in memory;

[0040] Step S2: Select a portion of all sensor data for testing, record the number of times each RDD is depended on (N) and the computation time (T), and calculate the cost of each RDD. Store the values ​​of all RDDs in a one-dimensional array C and sort all the values.

[0041] Step S3: Test all sensor data. When an RDD is used, update the number of times the RDD is depended on, N=N-1, and discard RDDs with a number of times they are depended on, 0.

[0042] Step S4: When memory is insufficient, evict the RDD with the lowest generation value in the cache according to the sorting result and update C.

[0043] The beneficial effects of the present invention include at least the following:

[0044] 1. After generating frequent itemsets for new data, the similarity is used to determine whether to merge or add them to the global candidate itemset list. This avoids frequent large-scale calculations on the entire dataset, reduces the consumption of computing resources, and improves the real-time performance and efficiency of data processing.

[0045] 2. By dynamically adjusting the joint distribution parameters and the support of frequently updated itemsets based on the updated global candidate itemset list, the data mining model can be updated and optimized in a timely manner, ensuring a rapid response to data changes.

[0046] 3. By calculating the marginal distribution of each sensor's data and the correlation coefficient between sensor data, a joint distribution of different sensor data is constructed. This fully considers the correlation and mutual influence between the sensor data, which helps to understand the information reflected by the sensor data of the unmanned underwater vehicle more comprehensively and accurately, and to uncover the complex relationships and patterns hidden behind the data, thereby providing a basis for more accurate data analysis and decision support. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0048] Figure 2 This is a flowchart illustrating the RDD transformation process according to an embodiment of the present invention.

[0049] Figure 3 This is a flowchart illustrating the caching strategy of an embodiment of the present invention;

[0050] Figure 4 This is a comparison chart of the Copula function under different parameters in the embodiments of the present invention;

[0051] Figure 5 This is a schematic diagram comparing the performance of the caching strategy and the LRU strategy in an embodiment of the present invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0053] like Figure 1 As shown, this embodiment of the invention provides an incremental frequent itemset mining method for unmanned underwater vehicles, including the following steps:

[0054] Step S1: Acquire data from multiple sensors of the unmanned underwater vehicle, perform frequent itemset mining on the data from all sensors, generate initial frequent itemsets for each sensor, and merge all initial frequent itemsets as a global candidate itemset list.

[0055] Specifically, the original sensor dataset is read from the UUV distributed storage system using a text file (TextFile), and the sensor dataset is divided into multiple scale levels based on multi-scale theory.

[0056] For example, sensor data can be divided into segments using a 1-hour timescale and each sensor as the dividing criterion:

[0057] Scale dataset 1: Sonar data from the past hour.

[0058] Scale dataset 2: Navigation data from the past hour.

[0059] Scale dataset 3: Dynamical system data for the past hour.

[0060] like Figure 2The diagram illustrates the RDD transformation process, demonstrating how unmanned underwater vehicle (UUV) sensor data is converted into a resilient distributed dataset (RDD) for frequent itemset mining. The steps include: reading the original sensor dataset from the UUV's distributed storage system using a text file (TextFile); dividing the dataset into multiple baseline-scale datasets based on sensor type using multi-scale theory; generating a corresponding resilient distributed dataset (RDD) for each baseline-scale dataset, denoted as TransactionRDD, and storing it in memory; performing frequent itemset mining on each TransactionRDD using the FP-Growth algorithm to obtain candidate frequent itemsets for each baseline-scale dataset; calculating the support of these candidate frequent itemsets and selecting those with support greater than a set threshold as the initial frequent itemsets for each baseline scale, corresponding to the resilient distributed dataset BSDSFrequentItemRDD for the baseline-scale datasets; and finally, merging the mining results from different baseline scales to form a global candidate itemset list (can_list), denoted as CandidataItemRDD, laying the foundation for subsequent incremental updates and global frequent itemset selection.

[0061] The FP-Growth algorithm is used to mine frequent itemsets in the sensor dataset at each scale level. Jaccard similarity is used to measure the support of each frequent itemset, and frequent itemsets with support greater than a set threshold are selected as the initial frequent itemsets for each scale. The similarity between frequent itemsets at each scale is calculated to prepare for subsequent incremental updates. The expression for calculating the support of frequent itemsets is:

[0062] ;

[0063] In the formula, For candidate frequent itemsets Support level; For candidate frequent itemsets The number of times it appears; This represents the total number of transactions.

[0064] Step S2: Calculate the edge distribution of data from each sensor and the correlation coefficient between the data, and construct the joint distribution of the data based on the edge distribution and the correlation coefficient.

[0065] Specifically, a binary joint probability density function model of sensor data is constructed based on Copula theory, separating the marginal distribution and dependency structure of the sensor data into a separate model. The specific steps are as follows:

[0066] Step S21: Calculate the marginal probability density function of sensor data such as sonar and navigation using kernel density estimation (KDE):

[0067] ;

[0068] ;

[0069] In the above formula, For sensor data The marginal probability density; This refers to the number of samples collected from the sensor data. The bandwidth for kernel density estimation; For kernel functions; Let be the observed value of the i-th sensor data point.

[0070] in The kernel function controls the smoothness of the estimated curve; a larger value results in a smoother curve, but may lose detail. It can be used to minimize the mean square error of integration (MISE) through cross-validation, or adaptively adjusted based on the sensor sampling frequency in UUV scenarios. The kernel function is used to weight neighboring data points, utilizing historical data... Estimating sensor variables The probability density distribution.

[0071] Step S22: Select the type of Copula function based on the dependency characteristics between variables. In this embodiment of the invention, Gaussian Copula is used to capture the conventional linear correlation of sensor data, and Clayton Copula is used to capture the asymmetric dependency under extreme conditions. The expression for calculating the correlation coefficient between sensor data using the Copula function is as follows:

[0072] ;

[0073] ;

[0074] ;

[0075] In the above formula, For Copula functions; , The probability distribution values ​​obtained after transforming sensor data x and y using the marginal distribution function; The correlation coefficient between sensor data x and y. This describes the linear correlation between sensor data; for The bivariate standard normal distribution function; The marginal distribution value is the inverse function of the standard normal distribution. , Convert to a standard normal variable; These are the dependency parameters for the Clayton Copula function.

[0076] in, The Clayton Copula function is used to model symmetric dependencies between sensor data. A larger Clayton Copula value indicates a stronger lower-tail dependency between two sensor data sets. For example, it's used to model sensor interoperability at extremely low values. If sonar signal strength and power system temperature are highly correlated at low values, then the Clayton Copula function is chosen and a larger value is estimated. .

[0077] Step S23: Using historical sensor data Estimate the parameters of the Copula function, specifically by calculating the parameters of the Copula function through maximum likelihood estimation:

[0078] ;

[0079] ;

[0080] In the above formula, Let be the maximum likelihood estimation function; , These are the parameters of the Copula function; This represents the number of samples in the historical sensor data. For Copula functions; , For historical sensor data The marginal distribution.

[0081] Step S3: Update the global candidate itemset list: Generate frequent itemsets for the new data from the sensors of the unmanned underwater vehicle during navigation, calculate the similarity between the new frequent itemsets and all the initial frequent itemsets, and if the similarity is greater than the set similarity threshold, merge the new frequent itemset into the corresponding initial frequent itemset; otherwise, add the new frequent itemset as a new itemset to the global candidate itemset list.

[0082] Specifically, when itemsets appearing in the new data intersect with existing itemsets, the support of frequent itemsets may exceed a critical threshold due to changes in incremental samples, or the historical support of a frequent itemset is close to a threshold and may become a frequent or infrequent itemset after updating, specific methods are needed to handle this. For example, maintaining a support boundary zone itemset index table, or adding itemsets in the incremental data whose frequency exceeds a warning threshold to a recount queue, and then labeling the affected itemsets in the global candidate itemset list to initiate the recount process. In this process, the FP-growth algorithm is used to obtain frequent itemsets from different groups of new data, and the similarity between the new data and the original dataset at the baseline scale is calculated. The frequent itemset is then compared with the initial frequent itemset BSi of the corresponding i-th baseline scale dataset in the global candidate itemset list to find the intersection and difference, and the global candidate itemset list is updated.

[0083] For unknown count estimation, frequent itemsets are updated by comparing the new dataset (D2) and the original dataset (D1), based on similarity calculations. The comparison of D1 and D2 involves set operations, using formulas including intersection D1∩D2, difference D1−D2, and D2−D1.

[0084] Step S4: Dynamically adjust the parameters of the joint distribution based on the updated global candidate itemset list, update the support of frequent itemsets based on the updated joint distribution, and thus update the global candidate itemset list.

[0085] Specifically, after the UUV sensor dataset is updated, based on the new data... Update the edge distribution of the sensor data, and adjust the parameters of the Copula function based on the updated edge distribution:

[0086] ;

[0087] In the formula, The correlation coefficient between the updated sensor data x and y; This represents the parameter that maximizes the objective function. The value of ; This represents the raw correlation coefficient between sensor data x and y.

[0088] The joint distribution of the sensor data is updated using the updated correlation coefficients, and the support of frequent itemsets is recalculated.

[0089] By following the above process, only the changed parameters need to be updated, avoiding repeated scanning of the entire dataset. The parameters updated during this process are shown in Table 1.

[0090] Table 1 Updated parameters

[0091]

[0092] In this embodiment of the invention, to meet the real-time data mining requirements of the UUV system, parallel computing is performed using the distributed computing framework (Spark), while a low-cost secondary execution caching strategy (LCDERP) is applied to optimize the use of computing resources. The specific steps are as follows:

[0093] S1: Use TextFile to read the UUV sensor dataset from the distributed file system HDFS, convert each benchmark scale dataset into an RDD, denoted as Transaction Resilient Distributed Dataset TransactionRDD, and store it in memory.

[0094] S2: According to the concept-based hierarchical grouping strategy, TransactionRDD is assigned to different groups. The number of groups is determined by the number of benchmark scale datasets. Each partition runs one benchmark scale dataset.

[0095] S3: Each group uses the FP-growth method to obtain the frequent itemsets of its benchmark scale dataset, which is denoted as the elastic distributed dataset BSDSFrequentItemRDD of the frequent itemsets of the benchmark scale dataset.

[0096] S4: Merge the mining results from different groups to obtain a global candidate set list can_list (<m,itemsets,i,count,sum> ), denoted as CandidataItemRDD, where m represents the hierarchical identifier of the baseline scale, i.e., the sensor type; itemsets represents the set of frequent itemsets, i.e., the set of items that meet the support threshold mined from sensor data; i represents the index of the baseline scale dataset to which the frequent itemset belongs, used to locate the source dataset of the itemset; count represents the number of times the frequent itemset appears in the corresponding baseline scale dataset; and sum represents the cumulative number of times the frequent itemset appears globally.

[0097] S5: Estimate the unknown itemset and update can_list.

[0098] S6: Set a support threshold min_sup, and filter frequent itemsets with support greater than min_sup, storing them in the frequent itemset list FI_list.<itemset,count> ), denoted as Target Scale Frequent Itemset Resilient Distributed Dataset TSFrequentItemRDD, where itemset represents a frequent itemset.

[0099] S7: Convert the newly added datasets of different benchmark scales into RDDs and assign them to the corresponding groups.

[0100] S8: Use the FP-growth method to obtain frequent itemsets for each group of new data. After obtaining the frequent itemsets of a certain benchmark scale dataset, calculate the similarity between the new data and the original dataset of that benchmark scale. Compare the frequent itemsets with the corresponding BSi in can_list, find the intersection and difference, and update can_list.

[0101] S9: Based on the updated sum value in can_list, filter out frequent itemsets that meet the target scale.

[0102] By using the above methods and parallelizing the processing of datasets at various benchmark scales using the Spark framework, computation time and memory consumption can be reduced, thereby ensuring the computational efficiency of UUVs in resource-constrained environments.

[0103] The low-cost secondary execution caching strategy LCDERP of this invention can effectively avoid recalculating intermediate results when memory is insufficient, thereby improving computational efficiency. Figure 3 As shown, the method includes the following steps:

[0104] S1: Select a portion of all sensor data for testing and record the number of times each RDD is depended upon. and calculation time Calculate the cost of each RDD. Store the cost of all RDDs in a one-dimensional array. In the middle, and sort all the values;

[0105] S3: Test all sensor data and update the number of times the RDD is depended on when it is used. Eliminate RDDs that have been depended on 0 times;

[0106] S4: When memory is insufficient, evict the RDD with the lowest value in the cache based on the sorting result and update the cache. .

[0107] To verify the effectiveness of the method provided in this embodiment of the invention, a comparative experiment was conducted based on UUV sensor data:

[0108] Experimental design: 100,000 transaction data were processed in 5 batches, and the method of the present invention was compared with that of traditional FP-Growth.

[0109] Results: Single batch processing time decreased from 120s to 40s, a reduction of 67%; total processing time decreased from 600s to 220s, a reduction of 63%. The parallel speedup ratio is close to linear, which can effectively meet the real-time requirements of UUV.

[0110] Anomaly detection: By injecting 5% of abnormal sensor data, the recall rate of the method in this embodiment of the invention is increased to 95%, while the traditional method is only 82%, and the false alarm rate is reduced to 7%, while the traditional method is 18%.

[0111] After dynamically adjusting the edge distribution and Copula parameters, the parameter update time is only 30% of that of the traditional method; the proportion of recalculation of affected itemsets is reduced by 70%, and global scanning is avoided.

[0112] like Figure 4 The parameters of the Clayton Copula function are shown below. A comparison chart showing the relationship between recall rate and low-value anomaly linkage. Low-value anomaly linkage refers to an abnormal state where two or more sensor data simultaneously show extremely low values. Recall rate measures the proportion of this abnormal pattern that is successfully identified. From Figure 4 As can be seen from this, when the parameters of Clayton Copula... When set to 2.5, the recall rate of low-value anomaly linkages improved by 25% compared to other parameter values ​​or traditional methods, indicating that this parameter value can more accurately capture the correlation of sensor data at extremely low values, enhancing the sensitivity to abnormal states of underwater vehicles. This result validates the advantages of the Clayton Copula function in modeling asymmetric lower-tail dependencies, which can be improved by adjusting the parameters. The ability of the optimizable function to identify linkage patterns with extremely low values.

[0113] like Figure 5 As shown, after adopting the LCDERP strategy, the RDD eviction efficiency is improved by 50% when memory is insufficient, and the cost calculation time is <0.1ms / time. The measured cache hit rate is 25% higher than that of the LRU strategy.

[0114] This invention addresses the issue of limited resources in UUV data processing by proposing an incremental frequent itemset mining method for UUV. By combining a multi-scale incremental frequent itemset mining algorithm with parallel computing technology, it optimizes the use of computational resources and significantly improves data mining efficiency under limited UUV data processing resources. First, this invention introduces multi-scale theory to hierarchically partition the UUV sensor dataset and constructs a joint distribution model based on Copula theory to separate edge and dependency. Incremental data fusion is achieved by dynamically updating the edge distribution and Copula parameters, avoiding repeated scanning of the entire dataset and reducing computational resource consumption. Furthermore, this invention integrates the Spark parallel computing framework for distributed data processing, effectively improving the speed and efficiency of UUV data mining. Through reasonable resource scheduling and memory management strategies, it ensures efficient memory usage during the incremental frequent itemset mining process, reducing data redundancy and computational overhead.

[0115] To address the computational complexity issue in large-scale UUV data mining, an optimization method combining an incremental frequent itemset mining algorithm with a low-cost secondary execution caching strategy (LCDERP) is proposed. This method accelerates frequent itemset mining by partitioning and parallelizing the UUV sensor dataset, using a distributed computing framework to distribute datasets of different baseline scales across different computing nodes for parallel processing. Simultaneously, the low-cost caching strategy prioritizes caching lower-cost RDDs in memory, avoiding redundant computation and data loss, further improving the algorithm's performance and efficiency to meet the needs of real-time UUV data analysis and fault prediction.

[0116] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described; only preferred embodiments of the present invention are illustrated. The descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. As long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0117] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this invention should be determined by the appended claims.

Claims

1. An incremental frequent itemset mining method for unmanned underwater vehicles, characterized in that, Includes the following steps: Step S1: Acquire data from multiple sensors of the unmanned underwater vehicle, perform frequent itemset mining on the data from all sensors, generate initial frequent itemsets for each sensor, and merge all initial frequent itemsets as a global candidate itemset list; Step S2: Calculate the edge distribution of data from each sensor and the correlation coefficient between the data, and construct the joint distribution of the data based on the edge distribution and correlation coefficient; Step S3: Update the global candidate itemset list: Generate frequent itemsets for the new data from the sensors of the unmanned underwater vehicle during navigation, calculate the similarity between the new frequent itemsets and all the initial frequent itemsets, and if the similarity is greater than the set similarity threshold, merge the new frequent itemsets into the corresponding initial frequent itemsets; otherwise, add the new frequent itemsets as a new itemset to the global candidate itemset list. Step S4: Dynamically adjust the parameters of the joint distribution based on the updated global candidate itemset list, update the support of frequent itemsets based on the updated joint distribution, and thus update the global candidate itemset list.

2. The incremental frequent itemset mining method for unmanned underwater vehicles according to claim 1, characterized in that: The step S1 of generating the initial frequent itemsets for each sensor includes the following steps: performing frequent itemset mining on the data of all sensors to generate candidate frequent itemsets for each sensor, calculating the support of all candidate frequent itemsets, and using the candidate frequent itemsets with support greater than a set threshold as the initial frequent itemsets for each sensor.

3. The incremental frequent itemset mining method for unmanned underwater vehicles according to claim 2, characterized in that: The expression for calculating the support is: ； In the formula, For candidate frequent itemsets Support level; For candidate frequent itemsets The number of times it appears; This represents the total number of transactions.

4. The incremental frequent itemset mining method for unmanned underwater vehicles according to claim 1, characterized in that: The expression for calculating the edge distribution of the sensor data in step S2 is: ； ； In the formula, For sensor data The marginal distribution; For sensor data The marginal probability density; This refers to the number of samples collected from the sensor data. The bandwidth for kernel density estimation; For kernel functions; Let be the observed value of the i-th sensor data point.

5. The incremental frequent itemset mining method for unmanned underwater vehicles according to claim 1, characterized in that: The expression for calculating the correlation coefficient between the data in step S2 is: ； ； ； In the above formula, For Copula functions; , These are the edge distribution values ​​of the sensor data x and y; The correlation coefficient between sensor data x and y; for The bivariate standard normal distribution function; It is the inverse function of the standard normal distribution; These are the dependent parameters of the Copula function.

6. The incremental frequent itemset mining method for unmanned underwater vehicles according to claim 1, characterized in that: In step S2, when constructing the joint distribution of the data, the parameters of the joint distribution are estimated using historical data from the sensors. The expression for estimating the parameters of the joint distribution is as follows: ； ； In the above formula, Let be the maximum likelihood estimation function; , These are the parameters of the Copula function; This represents the number of samples in the historical sensor data. For Copula functions; , For historical sensor data The marginal distribution.

7. The incremental frequent itemset mining method for unmanned underwater vehicles according to claim 1, characterized in that: In step S2, the expression for constructing the joint distribution of the data based on the marginal distribution and correlation coefficient is as follows: ； In the formula, The joint distribution of sensor data x and y; , These represent the edge distributions of sensor data x and y, respectively. The correlation coefficient between sensor data x and y; , These are the marginal probability densities of sensor data x and y, respectively.

8. The incremental frequent itemset mining method for unmanned underwater vehicles according to claim 1, characterized in that: The expression for calculating the similarity between frequent itemsets in step S3 is: ； In the formula, Jaccard similarity; , Represents sensor data; This represents the frequent itemsets in the sensor data D.

9. The incremental frequent itemset mining method for unmanned underwater vehicles according to claim 1, characterized in that: The expression for dynamically adjusting the parameters of the joint distribution based on the updated global candidate set list in step S4 is: ； In the formula, The correlation coefficient between the updated sensor data x and y; This represents the parameter that maximizes the objective function. The possible values ​​of ; For newly added sensor data; , These represent the edge distributions of sensor data x and y, respectively. This is the correlation coefficient between sensor data x and y.

10. A caching strategy applicable to the incremental frequent itemset mining method for unmanned underwater vehicles as described in any one of claims 1-9, characterized in that, Includes the following steps: Step S1: Use a distributed computing framework to convert sensor data into a resilient distributed dataset RDD, and store all RDDs in memory; Step S2: Select a portion of all sensor data for testing, record the number of times each RDD is depended on (N) and the computation time (T), and calculate the cost of each RDD. Store the values ​​of all RDDs in a one-dimensional array C and sort all the values. Step S3: Test all sensor data. When an RDD is used, update the number of times the RDD is depended on, N=N-1, and discard RDDs with a number of times they are depended on, 0. Step S4: When memory is insufficient, evict the RDD with the lowest generation value in the cache according to the sorting result and update C.

Images