Adaptive optimization Gaussian mixture modeling method
Through the adaptive optimization of Gaussian hybrid modeling method, the average comprehensive information criterion and EM algorithm are used to automatically determine the optimal component of GMM, which solves the problems of high computational complexity and poor model fitting caused by manual components, and realizes the problem of high computational complexity and poor model fitting. More efficient model fit and robustness.
Patent Information
- Application Number
- CN202410059913.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-15
- Publication Date
- 2024-12-13
AI Technical Summary
Existing Gaussian mixed models (GMMs) require manual specification when determining the optimal component, resulting in high computational complexity and prone to underfitting or overfitting problems.
Using the average comprehensive information criterion, the optimal component is automatically determined through adaptive optimization methods, and using technologies such as EM algorithm and Mahayana distance, set the significance interval [βmin, βmax] to avoid manually specifying the component and realize the adaptive interval. Determination of optimal components.
The computational complexity is reduced, underfitting and overfitting problems are avoided, and the robustness and efficiency of the model are improved.
Smart Images
Figure CN119128561A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of signal processing, and in particular to an adaptive optimization Gaussian mixture modeling method. Background Art
[0002] Gaussian mixture model (GMM) is a simple and effective modeling method, which has been successfully and widely used in machine learning, cluster analysis, pattern recognition, edge smoothness of images, image matching, statistical analysis, approximation of non-Gaussian noise and other fields. Selecting the appropriate number of Gaussian components M can avoid both underfitting and overfitting, so it is very important for applying Gaussian functions to establish models. However, manually selecting the number of components is neither reasonable nor scientific. Therefore, according to the Mahalanobis distance of the given data and the significance interval [β min ,β max ], accurately and reasonably locate the interval of the number M Very important.
[0003] Due to its global convergence, low computational complexity per iteration, storage economy and easy implementation, EM has become the most widely used method for finding the maximum likelihood estimate of GMM. However, researchers have proposed many model selection criteria to determine the optimal number of components for the EM algorithm when fitting GMM. A standardized method is provided to balance sensitivity and specificity through a log-likelihood function with a simple penalty, such as the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC). However, when applying these information criteria, it is still necessary to first specify an initial number of components M, and the problem of manually specifying the number of M components cannot be avoided. Therefore, an adaptive optimization Gaussian mixture modeling method is provided. Summary of the invention
[0004] The purpose of the present invention is to provide an adaptive optimization Gaussian mixture modeling method, which uses the average comprehensive information criterion to achieve the adaptive interval Best ingredients within The adaptive optimal determination solution can reduce the computational complexity of existing methods while avoiding underfitting and overfitting problems.
[0005] To achieve the above object, the present invention provides an adaptive optimization Gaussian mixture modeling method, comprising the following steps:
[0006] S1: Select a d-dimensional data set with a sample size of N As the experimental object, set the control parameter β and initialize it as follows:
[0007] μ=x1
[0008]
[0009]
[0010] sp=1
[0011] π=1
[0012] In the above formula, μ is the initial mean value, is the inverse covariance matrix, |∑1| -1 is the inverse determinant of the covariance matrix, sp is the cumulative posterior probability, and π is the prior probability;
[0013] S2: Start the calculation from n=2, and make a judgment on the size of n and N. As the data stream is input, calculate [d,M]:
[0014]
[0015] Calculate the square of the Mahalanobis distance d x,m , the formula is as follows:
[0016]
[0017] In the above formula, d x,m is the square of the Mahalanobis distance from sample x to the mth component, μ m is the mean of the mth component, ∑ m is the covariance matrix of the mth component. According to the results of the Mahalanobis distance square, the situations are classified as follows:
[0018] When the input data point meets the condition Then the update process is carried out;
[0019] When the input data point meets the condition Then the creation process is carried out;
[0020] In the above formula, represents the chi-square distribution with d degrees of freedom, 1-β is the probability corresponding to the chi-square distribution, and M is the number of Gaussian components;
[0021] After the update process or creation process, n=n+1, and repeat the above process until n>N, all data points are updated or created, and the adaptive interval is output.
[0022] S3: According to the adaptive interval The number of components I in the equation is used to calculate the regularized log-likelihood function S of the Gaussian mixture model. a [I,N], get the optimal number of components The expression is as follows:
[0023]
[0024] In the above formula, is the optimal number of components, N is the sample size;
[0025] S4: Based on the optimal amount of ingredients Determine The mixture model of Gaussian components is as follows:
[0026]
[0027] In the above formula, π t represents the prior probability of the tth component, μ t is the mathematical expectation of the tth component, ∑ t -1 is the inverse covariance matrix.
[0028] Preferably, in step S1, the value range of the control parameter β is [β min ,β max ], β min =0.1 11 and β max =0.1 2 .
[0029] Preferably, in step S2, the updating process is as follows:
[0030] Calculate x n The probability density of the kth component in the Gaussian distribution is:
[0031]
[0032] In the above formula, ∑ k is the covariance matrix of component k, and other specific parameters are calculated and updated as follows:
[0033]
[0034]
[0035]
[0036]
[0037]
[0038] μ k =μ k +ω k (X n -μ k )
[0039]
[0040] In the above formula, is the posterior probability of the kth component, sp k is the recalculated cumulative posterior probability, π k is the prior probability calculated during the iteration, ω k is the posterior ratio, is the inverse covariance matrix after iterative update, |∑ k | -1 is the inverse determinant of the covariance matrix, μ k is the mathematical expectation of the kth component, π m is the prior probability of the final output when the iteration is completed.
[0041] Preferably, in step S2, the creation process is as follows:
[0042] Create a new component M=M+1, calculated as follows:
[0043]
[0044]
[0045] μ M =x n
[0046] sp M =1
[0047]
[0048] In the above formula, μ M are the inverse covariance matrix and the mean of the covariance matrix of the new component M, sp M is the posterior probability initialization value, π m is the recalculated prior probability.
[0049] Preferably, in step S3, the optimal number of components is calculated The expression process is as follows:
[0050] Based on the EM algorithm, a mixed model with I Gaussian components is established to fit the data. The formula is as follows:
[0051]
[0052] The regularized log-likelihood function of the Gaussian mixture model is calculated using the mixture model fitting data as follows:
[0053]
[0054] In the above formula, K *The expression is as follows:
[0055]
[0056] In the above formula, I is the adaptive interval The number of components in K * is the mean of eight classical information criteria.
[0057] Therefore, the present invention adopts an adaptive optimization Gaussian mixture modeling method of the above method, which has the following advantages:
[0058] (1) In the present invention, since EM has the characteristics of global convergence and low computational complexity of each iteration, it can be effectively used in the maximum likelihood estimation of Gaussian fitting, which can effectively reduce the computational complexity and reduce the computational cost.
[0059] (2) In the present invention, by setting the significance interval [β min ,β max ], and then locate the adaptive component number interval of the component number M The underfitting and overfitting problems caused by manually selecting the number of components M are avoided.
[0060] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] Figure 1 A flowchart of an adaptive optimization Gaussian mixture modeling method of the present invention;
[0062] Figure 2 A schematic diagram of an adaptive optimization Gaussian mixture modeling method of the present invention in which M is less than 8 in a data set 1 with 8 components;
[0063] Figure 3 A schematic diagram of an adaptive optimization Gaussian mixture modeling method of the present invention in which M is greater than 8 in a data set 1 with 8 components;
[0064] Figure 4 A schematic diagram of an adaptive optimization Gaussian mixture modeling method of the present invention in which M is less than 6 in a data set 2 with 6 components;
[0065] Figure 5 A schematic diagram of an adaptive optimization Gaussian mixture modeling method of the present invention in which M is greater than 6 in a data set 2 with 6 components;
[0066] Figure 6 A schematic diagram of a self-adaptive optimized Gaussian mixture modeling method according to the present invention in which M is less than 4 in a data set 3 having 4 components;
[0067] Figure 7A schematic diagram of an adaptive optimization Gaussian mixture modeling method of the present invention in which M is greater than 4 in a data set 3 with 4 components;
[0068] Figure 8 A schematic diagram of an adaptive optimization Gaussian mixture modeling method of the present invention in which M is less than 6 in a data set 4 with 6 components;
[0069] Fig. 9 A schematic diagram of an adaptive optimization Gaussian mixture modeling method of the present invention in which M is greater than 6 in a data set 4 with 6 components;
[0070] Fig.10 A scatter plot of seven heart disease data features generated in an adaptive optimization Gaussian mixture modeling method of the present invention;
[0071] Fig.11 In an adaptive optimization Gaussian mixture modeling method of the present invention, different information criteria are used to optimize GMM to obtain the component number distribution of the optimal number of components;
[0072] Fig.12 The invention provides a method for adaptively optimizing Gaussian mixture modeling. The corresponding times are shown in Figure C;
[0073] Fig.13 This is a schematic diagram of 10-fold cross validation in an adaptively optimized Gaussian mixture modeling method of the present invention. DETAILED DESCRIPTION
[0074] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations. The specific model specifications need to be selected and determined according to the actual specifications of the device, and the specific selection calculation method adopts the existing technology in the field, so it will not be described in detail.
[0075] Example
[0076] like Figure 1 The method disclosed in the present invention is an adaptive optimization Gaussian mixture modeling method, comprising the following steps:
[0077] S1: Select a d-dimensional data set with a sample size of N As the experimental object, the control parameter β is set, and the value range of the control parameter β is [β min ,β max ], β min =0.111 and β max =0.1 2 , setting a reasonable input parameter β value is an important factor in controlling the number of output components M, and initialization is created as follows:
[0078] μ=x1
[0079]
[0080]
[0081] sp=1
[0082] π=1
[0083] In the above formula, μ is the initial mean value, is the inverse covariance matrix, |∑1| -1 is the inverse determinant of the covariance matrix, sp is the cumulative posterior probability, and π is the prior probability;
[0084] S2: Start the calculation from n=2, and make a judgment on the size of n and N. As the data stream is input, calculate d, M]:
[0085]
[0086] Calculate the square of the Mahalanobis distance d x,m , the formula is as follows:
[0087]
[0088] In the above formula, d x,m is the square of the Mahalanobis distance from sample x to the mth component, μ m is the mean of the mth component, ∑ m is the covariance matrix of the mth component. According to the result of the Mahalanobis distance square, the situation is divided into the creation process and the update process. The specific process is as follows:
[0089] When the input data point meets the condition Then the update process is carried out, and the update process is as follows:
[0090] Calculate x n The probability density of the kth component in the Gaussian distribution is:
[0091]
[0092] In the above formula, ∑ k is the covariance matrix of component k; other specific parameters are calculated and updated as follows:
[0093]
[0094]
[0095]
[0096]
[0097]
[0098] μ k =μ k +ω k (x n -μ k )
[0099]
[0100] In the above formula, is the posterior probability of the kth component, π k is the prior probability calculated during the iteration <sp k is the recalculated cumulative posterior probability, ω k is the posterior ratio; is the inverse covariance matrix after iterative update, |∑ k | -1 is the inverse determinant of the covariance matrix, μ k is the mathematical expectation of the kth component, π m is the prior probability of the final output when the iteration is completed.
[0101] When the input data point meets the condition Then create it, the creation process is as follows:
[0102] Create a new component M=M+1, calculated as follows:
[0103]
[0104]
[0105] μ M =x n
[0106] sp M =1
[0107]
[0108] in, μ M are the covariance matrix and the mean of the covariance matrix of the new component M, sp M is the posterior probability initialization value, π m is the recalculated prior probability.
[0109] According to the meaning of the parameter 1-β, we know that: ① A smaller β will make more sample points meet the iteration conditions and thus be used for updating existing components, while fewer sample points are used to create new Gaussian components, thus generating a smaller number of components M; ② A larger β will make a larger number of sample points meet the creation conditions and thus create new components, while fewer sample points are used for component iteration, thus generating a larger number of components M. Therefore, by specifying a reasonable smaller value as β = β min and larger β=β max The number of adaptive smaller components can be obtained respectively and adaptively large number of components Then constitute the adaptive component quantity interval
[0110] After the update process or creation process, n=n+1, and repeat the above process until n>N, that is, all data points are updated or created, and the adaptive interval is output.
[0111] Table 1 Specific parameters of the four simulation data sets
[0112]
[0113] Combine the above methods to conduct experiments, such as Figure 2-9 , four simulation data sets are calculated, and the parameters of the four simulation data sets are shown in Table 1: According to the experimental results, the conclusions are as follows: ① Figure 2 , 4 , 6, and 8, as the β value increases from a larger value to β min Move, the distribution range of the output value becomes narrower and narrower, which reduces the number of times M approaches the optimal number; ② such as Figure 3 , 5 , 7, 9 As the β value increases from a small value to β max The direction moves so that the number of times M approaches the optimal number decreases.
[0114] S3: According to the adaptive interval The number of components I in the equation is used to calculate the regularized log-likelihood function S of the Gaussian mixture model. a [I,N], calculate the optimal number of components The expression process is as follows:
[0115] Based on the EM algorithm, a mixed model with I Gaussian components is established to fit the data. The formula is as follows:
[0116]
[0117] The regularized log-likelihood function of the Gaussian mixture model is calculated using the mixture model fitting data formula as follows:
[0118]
[0119] In the above formula, K * The expression is as follows:
[0120]
[0121] In the above formula, I is the adaptive interval The number of components in K * is the mean of 8 classical information criteria. The optimal number of components is obtained The expression is as follows:
[0122]
[0123] In the above formula, is the optimal number of components, and N is the sample size.
[0124] S4: Based on the optimal amount of ingredients Determine The mixture model of Gaussian components is as follows:
[0125]
[0126] In the above formula, π t represents the prior probability of the tth component, μ t is the mathematical expectation of the tth component, ∑ t -1 is the inverse covariance matrix.
[0127] Experiments and evaluations are carried out in combination with real experimental data, including the following two parts: experiments on real heart sound datasets and classification performance evaluation.
[0128] Part I: Experiments on real heart sound datasets. Based on the Gaussian mixture model, real heart sound data of seven heart diseases (mitral regurgitation, mitral stenosis, aortic septal defect, normal heart sounds, aortic stenosis, aortic regurgitation and ventricular septal defect) are fitted to verify the feasibility of the adaptive optimization scheme. In order to increase the scale of the real dataset, the mvnrnd function is used in MATLAB 2018B to generate a simulated three-dimensional heart sound dataset with the same specified parameters. The specific parameters are shown in Table 2 below: 5000 samples are generated in each execution, and a total of 100 executions are performed. Experiments show that no matter what criterion is chosen to optimize the GMM, the results of the 7 components are the same, such as Figure 10-12 As shown, the corresponding Table 2 is formed as follows:
[0129] Table 2 Parameters of real three-dimensional heart sound data
[0130]
[0131] Part II: Classification Performance Evaluation
[0132] The method of the present invention is compared with classic clustering algorithms, such as random forest RF, neural network NN, linear support vector machine LSVM, radial basis function support vector machine RBFSVM and fast incremental Gaussian mixture model FIGMM, to perform classification comparative analysis on typical training sets, and the results are shown in Table 3 below:
[0133] Table 3 Accuracy of different algorithms (%)
[0134]
[0135] In order to avoid result deviation, the performance evaluation results were obtained by 10-fold cross validation. The process is summarized as follows:
[0136] ① The validation data set is randomly divided into 10 parts, consisting of 10 samples of equal size from the original data set, marked as the i-th (i=1,2,...,10) sample.
[0137] ② From the 10 samples, select the i-th sample as the test sample, and the remaining samples as training samples to calculate the overall accuracy OA of the currently selected sample (expressed as OA i ), used to validate the classifier.
[0138] ③The above process is repeated 10 times by selecting different samples as test samples. Fig.13 shown.
[0139] The final accuracy of the classifier is denoted as μ OA ±σ OA , the formula is as follows:,,
[0140]
[0141] The experimental results show that: ① Overall, the classification accuracy obtained by the method of the present invention is greater than that of FIGMM; ② On average, the performance of the algorithm of the present invention is better than that of the random forest algorithm.
[0142] Therefore, the present invention adopts an adaptive optimization Gaussian mixture modeling method, which uses the average comprehensive information criterion to achieve the adaptive interval Best ingredients within The adaptive optimal determination solution can reduce the computational complexity and avoid underfitting and overfitting problems.
[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.
Claims
1. An adaptive optimization Gaussian mixture modeling method, characterized in that: The following steps are involved: S1: Select a d-dimensional data set with a sample size of N As the experimental object, set the control parameter β and initialize it as follows: μ=x1 sp=1 π＝1 In the above formula, μ is the initial mean value, is the inverse covariance matrix, |∑1| -1 is the inverse determinant of the covariance matrix, sp is the cumulative posterior probability, and π is the prior probability; S2: Start the calculation from n=2, and make a judgment on the size of n and N. As the data stream is input, calculate d, M]: [d,M]=size(M), Calculate the square of the Mahalanobis distance d x,m , the formula is as follows: In the above formula, d x,m is the square of the Mahalanobis distance from sample x to the mth component, μ m is the mean of the mth component, ∑ m is the covariance matrix of the mth component. According to the results of the Mahalanobis distance square, the situations are classified as follows: When the input data point meets the condition Then the update process is carried out; When the input data point meets the condition Then the creation process is carried out; In the above formula, represents the chi-square distribution with d degrees of freedom, 1-β is the probability corresponding to the chi-square distribution, and M is the number of Gaussian components; After the update process or creation process, n=n+1, and repeat the above process until n>N, all data points are updated or created, and the adaptive interval is output. S3: According to the adaptive interval The number of components I in the equation is used to calculate the regularized log-likelihood function S of the Gaussian mixture model. a [I,N], get the optimal number of components The expression is as follows: In the above formula, is the optimal number of components, N is the sample size; S4: Based on the optimal amount of ingredients Determine The mixture model of Gaussian components is as follows: In the above formula, π t represents the prior probability of the tth component, μ t is the mathematical expectation of the tth component, ∑ t -1 is the inverse covariance matrix.
2. The method of adaptively optimizing Gaussian mixture modeling according to claim 1, characterized in that: In step S1, the value range of the control parameter β is [β min ,β max ], β min =0.1 11 and β max =0.1 2 .
3. The method of adaptively optimizing Gaussian mixture modeling according to claim 2, characterized in that: In step S2, the updating process is as follows: Calculate x n The probability density of the kth component in the Gaussian distribution is: In the above formula, ∑ k is the covariance matrix of component k, and other specific parameters are calculated and updated as follows: m k =μ k +oh k (x n -m k ) In the above formula, is the posterior probability of the kth component, sp k is the recalculated cumulative posterior probability, π k is the prior probability calculated during the iteration, ω k is the posterior ratio, is the inverse covariance matrix after iterative update, |∑ k | -1 is the inverse determinant of the covariance matrix, μ k is the mathematical expectation of the kth component, π m is the prior probability of the final output when the iteration is completed.
4. The adaptive optimization Gaussian mixture modeling method according to claim 3, characterized in that: In step S2, the creation process is as follows: Create a new component M=M+1, calculated as follows: m M =x n sp M =1 In the above formula, μ M are the inverse covariance matrix and the mean of the covariance matrix of the new component M, sp M is the posterior probability initialization value, π m is the recalculated prior probability.
5. The method of adaptively optimizing Gaussian mixture modeling according to claim 4, characterized in that: In step S3, the optimal number of components is calculated The expression process is as follows: Based on the EM algorithm, a mixed model with I Gaussian components is established to fit the data. The formula is as follows: The regularized log-likelihood function of the Gaussian mixture model is calculated using the mixture model fitting data as follows: In the above formula, K * The expression is as follows: In the above formula, I is the adaptive interval The number of components in K * is the mean of eight classical information criteria.