Mutual information acceleration estimation method based on sequence importance sampling

Abstract

The invention discloses a mutual information acceleration estimation method based on sequence importance sampling, which belongs to the field of mutual information acceleration, and comprises the following steps of: S1, converting a problem of solving the number of concatenated tables in mutual information calculation into a problem of solving the total number and the distribution probability of the concatenated tables; s2, solving the total number of matrixes under column constraint to obtain a column sum set and a row sum set; s3, approximating the real distribution probability by using the approximate distribution probability; s4, solving and optimizing the column number, and substituting the approximate distribution probability and the column sum set to obtain an approximate estimation result of the number of the column linked tables; and S5, taking the approximate estimation result as proposal distribution, and sampling through a column-by-column sampling strategy to obtain a final estimation result of the mutual information. By adopting the mutual information acceleration estimation method based on sequence importance sampling, the calculation efficiency is improved, the precision of a calculation result is ensured, and the balance between the efficiency and the precision is realized.

Description

Technical Field

[0001] The present invention relates to the field of mutual information acceleration technology, and in particular to a mutual information acceleration estimation method based on sequence importance sampling. Background Art

[0002] In the era of big data, where global data volumes are growing exponentially, the complex and diverse nature of data poses significant challenges to data analysis technologies. As a core pillar of machine learning and pattern recognition, variable correlation measurement plays a key role in feature selection, cluster analysis, and the interpretability of deep learning models. Mutual information (MI), a core quantitative tool in information theory, is valuable for scientifically characterizing the degree of dependence between two random variables by measuring the amount of information one variable provides about another.

[0003] Compared with traditional linear correlation analysis methods such as the Pearson correlation coefficient, Spearman rank correlation coefficient, and CorGC correlation coefficient, mutual information has the following significant technical advantages:

[0004] 1. Capability of capturing nonlinear dependencies: Breaking through the limitations of linear relationship assumptions, it can effectively characterize complex nonlinear interaction patterns between variables;

[0005] 2. Robustness advantage: It is naturally immune to outliers in the data, avoiding the interference of extreme values on dependency evaluation, and ensuring that the measurement results are close to the real data generation mechanism;

[0006] 3. Universality: It does not rely on specific data distribution assumptions and can maintain stable evaluation performance in both structured and unstructured data scenarios. It does not require complex data standardization preprocessing, significantly reducing data cleaning costs.

[0007] However, the engineering application of mutual information faces a fundamental limitation of computational complexity: its core computational process is equivalent to solving the problem of finding the number of contingency tables Ω(r,c) that satisfy given row and column constraints. Theoretical research has confirmed that this problem belongs to the #P-hard complexity category, meaning that exact calculation is not feasible at practical data scales, and approximate estimation methods must be relied upon. Although existing technologies have developed approximate algorithms such as maximum entropy estimation, Markov chain Monte Carlo estimation (MCMC), and linear time estimation, maximum entropy estimation based on simplifying assumptions struggles to preserve high-order dependencies. MCMC methods are limited by their convergence speed and are inefficient for large-scale data, while linear time estimation suffers from over-approximation and loss of accuracy. Therefore, existing technologies struggle to strike a balance between efficiency and accuracy, failing to meet the practical needs of real-time, accurate analysis in big data scenarios. Summary of the Invention

[0008] The purpose of the present invention is to provide a mutual information accelerated estimation method based on sequence importance sampling to solve the above technical problems.

[0009] To achieve the above object, the present invention provides a mutual information accelerated estimation method based on sequence importance sampling, comprising the following steps:

[0010] S1. Convert the problem of solving the number of contingency tables in mutual information calculation into the problem of solving the total number and distribution probability of contingency tables;

[0011] S2. Use permutation and combination theory to solve the total number of matrices under column constraints and obtain the column sum set and row sum set;

[0012] S3. Construct the true distribution probability based on rows and sets, introduce the optimized number of columns to replace the initial number of columns to construct the approximate distribution probability, and then use the approximate distribution probability to approximate the true distribution probability through variational inference;

[0013] S4. Use the moment matching method to make the mean and covariance of the approximate distribution probability consistent with the true distribution probability, solve the optimized number of columns, and substitute the approximate distribution probability and the column sum set, multiply the approximate probability distribution by the column sum set to obtain an approximate estimate of the number of columns in the contingency table;

[0014] S5. Use the approximate estimation result as the proposal distribution and sample it through the column-by-column sampling strategy to obtain the final estimation result of the mutual information.

[0015] Preferably, in step S1, it is assumed that A(c) is a non-negative m*n integer matrix X=(x ij ) set, set the following objective function:

[0016] Ω(r,c)=Pr(r∣c)∣A(c)∣ (1);

[0017] Where Ω(r,c) represents the number of contingency tables; Pr(r|c) represents the true distribution probability that the row sum is r given that the column sum is c.

[0018] Preferably, in step S2, the permutation and combination theory is used to transform each column c j The number of non-negative integer solutions to assign elements to m rows is Get the set A(c) whose column sum is c:

[0019]

[0020] Where n represents the number of columns in set a(c);

[0021] Use permutation and combination theory to transform each row r i The number of non-negative integer solutions to assign elements to n columns is Get the set A(r) whose row sum is r:

[0022]

[0023] Preferably, step S3 specifically includes the following steps:

[0024] S31. Based on the set A(r) with row sum r, construct the true distribution probability Pr(r|c):

[0025]

[0026] Where |A| represents a non-negative m*n integer matrix X=(x ij ) set; N represents the sum of matrix elements;

[0027] S32. Under the uniform distribution assumption, replace the initial number of columns n with the optimized number of columns α to construct the approximate distribution probability Pr(r|α):

[0028]

[0029] Preferably, in step S4, the approximate estimation result expression is as follows:

[0030]

[0031] Where Ω(r,c)' represents the approximate estimation result of Ω(r,c), N represents the total number of matrix elements, m represents the number of matrix rows, and α represents the number of optimized columns obtained by moment matching. The calculation formula is as follows:

[0032]

[0033] Preferably, in step S5, one column is sampled each time and the sum of the remaining rows and columns is updated until all columns are sampled. During this process, the sampling weight is calculated using the following formula:

[0034]

[0035] Where q(X) represents the sampling weight of the integer matrix X; Ω(r',c') represents the number of matrix contingency tables after removing the sampling columns;

[0036] Finally, the final estimation result of the mutual information is obtained by aggregating the importance weights to achieve accelerated estimation of the mutual information, and the formula is as follows:

[0037]

[0038] Where, Ω(r,c)” represents the final estimation result; q(X (i) ) represents the sampling weight of the i-th matrix element.

[0039] Therefore, the present invention adopts the above-mentioned mutual information accelerated estimation method based on sequence importance sampling, which has the following beneficial effects:

[0040] 1. Significantly improve computational efficiency: By simplifying the conditional probability distribution dimension through variational inference and utilizing the Sequential Importance Sampling (SIS) strategy, approximate distributions are used to replace the exact distributions that are difficult to directly calculate. This avoids the exponential computational complexity of the #P-hard problem, significantly reduces computational time, and meets the real-time requirements of big data scenarios.

[0041] 2. Balancing Accuracy and Efficiency: The moment matching method ensures that the mean and covariance of the approximate distribution are consistent with the true distribution, keeping the statistical characteristics close to the real situation and ensuring computational accuracy. It also reduces computational costs through approximation and sampling strategies, solving the core problem of balancing efficiency and accuracy in traditional algorithms.

[0042] 3. Reduce computational complexity: Introduce an optimized number of columns instead of the initial number of columns to transform the high-dimensional complex distribution into a low-dimensional controllable distribution. Combined with permutation and combination theory, the total number of column constraint matrices is accurately calculated. By decomposing and simplifying the problem, the mathematical complexity of mutual information estimation is fundamentally reduced.

[0043] In summary, the present invention adopts the hierarchical strategy of "problem decomposition - precise calculation - distribution approximation - moment matching optimization - sequence sampling", which significantly reduces the time complexity of mutual information estimation while ensuring calculation accuracy, and solves the imbalance problem between efficiency and accuracy of traditional algorithms.

[0044] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a flow chart of a mutual information accelerated estimation method based on sequence importance sampling described in the present invention. DETAILED DESCRIPTION

[0046] In order to make the purposes, technical solutions and advantages disclosed in the embodiments of the present invention clearer, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, where the same or similar numbers throughout represent the same or similar elements or elements with the same or similar functions.

[0047] It should be noted that the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products or devices.

[0048] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] like Figure 1 As shown, a mutual information acceleration estimation method based on sequence importance sampling includes the following steps:

[0050] S1. Convert the problem of solving the number of contingency tables in mutual information calculation into the problem of solving the total number and distribution probability of contingency tables;

[0051] In step S1, it is assumed that A(c) is a non-negative m*n integer matrix X=(x ij ) set, set the following objective function:

[0052] Ω(r,c)=Pr(r∣c)∣A(c)∣ (1);

[0053] Where Ω(r,c) represents the number of contingency tables; Pr(r|c) represents the true distribution probability that the row sum is r given that the column sum is c.

[0054] S2. Use permutation and combination theory to solve the total number of matrices under column constraints and obtain the column sum set and row sum set;

[0055] In step S2, the permutation and combination theory is used to transform each column c j The number of non-negative integer solutions to assign elements to m rows is Get the set A(c) whose column sum is c:

[0056]

[0057] Where n represents the number of columns in set A(c);

[0058] Use permutation and combination theory to transform each row r i The number of non-negative integer solutions to assign elements to n columns is Get the set A(r) whose row sum is r:

[0059]

[0060] S3. Construct the true distribution probability based on rows and sets, introduce the optimized number of columns to replace the initial number of columns to construct the approximate distribution probability, and then use the approximate distribution probability to approximate the true distribution probability through variational inference;

[0061] Step S3 specifically includes the following steps:

[0062] S31. Based on the set A(r) with row sum r, construct the true distribution probability Pr(r|c):

[0063]

[0064] Where |A| represents a non-negative m*n integer matrix X=(x ij ) set; N represents the sum of matrix elements;

[0065] S32. Under the uniform distribution assumption, replace the initial number of columns n with the optimized number of columns α to construct the approximate distribution probability Pr(r|α):

[0066]

[0067] S4. Use the moment matching method to make the approximate distribution probability consistent with the mean and covariance of the true distribution probability, solve the optimized number of columns, and substitute the approximate distribution probability and the column sum set to obtain an approximate estimate of the number of contingency tables;

[0068] In step S4, the approximate estimation result expression is as follows:

[0069]

[0070] Where Ω(r,c)' represents the approximate estimation result of Ω(r,c), N represents the total number of matrix elements, m represents the number of matrix rows, and α represents the number of optimized columns obtained by moment matching. The calculation formula is as follows:

[0071]

[0072] S5. Use the approximate estimation result as the proposal distribution and sample it through the column-by-column sampling strategy to obtain the final estimation result of the mutual information.

[0073] In step S5, one column is sampled each time and the sum of the remaining rows and columns is updated until all columns are sampled. During this process, the sampling weight is calculated using the following formula:

[0074]

[0075] Where q(X) represents the sampling weight of the integer matrix X; Ω(r',c') represents the number of matrix contingency tables after removing the sampling columns;

[0076] Finally, the final estimation result of the mutual information is obtained by aggregating the importance weights to achieve accelerated estimation of the mutual information, and the formula is as follows:

[0077]

[0078] Where, Ω(r,c)” represents the final estimation result; q(X (i) ) represents the sampling weight of the i-th matrix element.

[0079] Given an m*n matrix, where m=n=4, and the sets of row sums and column sums are represented by r and c respectively, and r=[220,215,93,64], c=[108,286,71,127], the matrix shown in Table 1 is obtained.

[0080] Table 1 Example matrix

[0081]

[0082] The example matrix in Table 1 is processed using the present invention and the traditional method, and the obtained results are shown in Table 2.

[0083] Table 2 Calculation results

[0084] method Calculation results The method of the present invention (no sampling) 34.77122 Method of the present invention (SIS sampling) 34.74119 Maximum entropy estimation 34.80192 G00dCrook estimate 34.77095 Actual value 34.7425

[0085] As shown in Table 2, the calculation results using the method described in the present invention are closest to the actual values, thus demonstrating the effectiveness of the present invention. It should be noted that the present invention can be applied to fields such as community detection and cluster analysis. When faced with an undivided community, the present invention can achieve more accurate and rapid classification, helping to promote research in fields such as data analysis.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solutions of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A mutual information acceleration estimation method based on sequence importance sampling, characterized by: The following steps are involved: S1. Convert the problem of solving the number of contingency tables in mutual information calculation into the problem of solving the total number and distribution probability of contingency tables; S2. Use permutation and combination theory to solve the total number of matrices under column constraints and obtain the column sum set and row sum set; S3. Construct the true distribution probability based on rows and sets, introduce the optimized number of columns to replace the initial number of columns to construct the approximate distribution probability, and then use the approximate distribution probability to approximate the true distribution probability through variational inference; S4. Use the moment matching method to make the mean and covariance of the approximate distribution probability consistent with the true distribution probability, solve the optimized number of columns, and substitute the approximate distribution probability and the column sum set, multiply the approximate probability distribution by the column sum set to obtain an approximate estimate of the number of columns in the contingency table; S5. Use the approximate estimation result as the proposal distribution and sample it through the column-by-column sampling strategy to obtain the final estimation result of the mutual information.

2. The method for accelerating mutual information estimation based on sequence importance sampling according to claim 1, characterized in that: In step S1, it is assumed that A(c) is a non-negative m*n integer matrix X=(x ij ) set, set the following objective function: Ω(r,c)=Pr(r∣c)∣A(c)∣ (1); Where Ω(r,c) represents the number of contingency tables; Pr(r|c) represents the true distribution probability that the row sum is r given that the column sum is c.

3. The method for accelerating mutual information estimation based on sequence importance sampling according to claim 2, characterized in that: In step S2, the permutation and combination theory is used to transform each column c j The number of non-negative integer solutions to assign elements to m rows is Get the set A(c) whose column sum is c: Where n represents the number of columns in set a(c); Use permutation and combination theory to transform each row r i The number of non-negative integer solutions to assign elements to n columns is Get the set A(r) whose row sum is r:

4. The method for accelerating mutual information estimation based on sequence importance sampling according to claim 3, characterized in that: Step S3 specifically includes the following steps: S31. Based on the set A(r) with row sum r, construct the true distribution probability Pr(r|c): Where |A| represents a non-negative m*n integer matrix X=(x ij ) set; N represents the sum of matrix elements; S32. Under the uniform distribution assumption, replace the initial number of columns n with the optimized number of columns α to construct the approximate distribution probability Pr(r|α):

5. The method for accelerating mutual information estimation based on sequence importance sampling according to claim 4, characterized in that: In step S4, the approximate estimation result expression is as follows: Where Ω(r,c)' represents the approximate estimation result of Ω(r,c), N represents the total number of matrix elements, m represents the number of matrix rows, and α represents the number of optimized columns obtained by moment matching. The calculation formula is as follows:

6. The method for accelerating mutual information estimation based on sequence importance sampling according to claim 5, characterized in that: In step S5, one column is sampled each time and the sum of the remaining rows and columns is updated until all columns are sampled. During this process, the sampling weight is calculated using the following formula: Where q(X) represents the sampling weight of the integer matrix X; Ω(r',c') represents the number of matrix contingency tables after removing the sampling columns; Finally, the final estimation result of the mutual information is obtained by aggregating the importance weights to achieve accelerated estimation of the mutual information, and the formula is as follows: Where, Ω(r,c)” represents the final estimation result; q(X (i) ) represents the sampling weight of the i-th matrix element.

Images