Gaussian mixture modeling method based on adaptive optimization correction incremental algorithm
Through adaptive optimization correction incremental algorithm and regression information criterion, the problem of complex component selection and high modeling time cost of Gaussian mixed model in cluster analysis is solved, and efficient model selection and calculation efficiency is achieved, avoiding the over-the-counter Fit and underfit.
Patent Information
- Application Number
- CN202411008569.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-25
- Publication Date
- 2024-11-26
AI Technical Summary
In cluster analysis, Gaussian mixed model has problems such as complex component selection, difficult model selection criteria, and high modeling time cost. Especially in high-dimensional and big data situations, existing methods cannot achieve adaptive optimization.
Using Gaussian hybrid modeling method based on adaptive optimization correction incremental algorithm, the data reading module, adaptive optimization module and Gaussian hybrid model creates module, proposes the regression information criterion RIC of adaptive data, and dynamically adjusts the parameters to select the optimal one Gaussian mixed model to reduce computational complexity.
Adaptive optimization of Gaussian mixed model is realized, which reduces the complexity of component selection and modeling time cost, improves the accuracy and calculation efficiency of model selection, and avoids overfitting and underfitting problems.
Smart Images

Figure CN119026695A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of the technical field of Gaussian mixture models, and in particular to a Gaussian mixture modeling method based on an adaptive optimization modified incremental algorithm. Background Art
[0002] Gaussian Mixture Model It has been widely used in fields such as pattern recognition, cluster analysis, machine learning, graphic matching, statistical analysis, noise estimation, etc. For cluster analysis problems, the Gaussian mixture model can be fitted using the M set of parameters {π m ,μ M ,∑ m |m=1,…,M} indicates that the clustering purpose is achieved.
[0003] However, in the field of Gaussian mixture model research, researchers at home and abroad also face many difficulties. First, in order to achieve high-quality clustering effects, the expectation maximization EM algorithm needs to give a cluster number M before calculation. Without any prior knowledge, M is specified, and the generated Gaussian mixture model inevitably has deviations, resulting in overfitting and underfitting problems in the clustering results. Furthermore, existing model evaluation criteria such as Akaike criterion AIC and Bayesian criterion BIC have certain limitations. Affected by the size and dimensionality of the data, there is no existing commonly used criterion that can achieve adaptive data and model selection. In addition, the computational complexity of using the EM algorithm to establish a Gaussian mixture model is high. In the case of high dimensions and big data, the calculation speed is slow and the convergence effect is poor. After updating the data set, it is impossible to perform adaptive optimization based on the existing model and needs to be remodeled.
[0004] In recent years, with the continuous development of big data resources and computer hardware and software, Gaussian mixture models based on the EM algorithm have been widely used in various fields, and optimization has been carried out to address existing problems, such as using GPU to accelerate the EM algorithm to accelerate modeling, and proposing new criteria based on the Bayesian criterion and the Akaike criterion. However, most of the existing methods are derivative products based on the original EM algorithm framework, and no reasonable solutions have been proposed for the problems mentioned above, such as high EM computational cost and poor adaptive data optimization modeling capabilities. Summary of the invention
[0005] The purpose of the present invention is to provide a Gaussian mixture modeling method based on an adaptive optimization modified incremental algorithm, which can solve the problems of complex component number selection, difficult model selection criteria and high modeling time cost in the process of establishing a Gaussian mixture model.
[0006] To achieve the above object, the present invention provides a Gaussian mixture modeling method based on an adaptive optimization modified incremental algorithm, comprising the following steps:
[0007] S1: Establish a Gaussian mixture model system based on the adaptive optimization modified incremental algorithm, including a data reading module, a Gaussian mixture model creation module and an adaptive optimization module;
[0008] The data reading module is responsible for reading the data set;
[0009] S2: Setting adjustable parameters for Gaussian mixture model creation module And build an adaptive optimization and correction incremental algorithm, combine the data set read by the data reading module, and establish a series of Gaussian mixture models;
[0010] S3: For the adaptive optimization module, the regression information criterion RIC of adaptive data is proposed, the size of the adaptive data set is adapted, and the optimal Gaussian mixture model in the series of Gaussian mixture models is selected.
[0011] Preferably, the process of the data reading module reading the data set is as follows: selecting data in a specified file format, selecting As an object, where N is the number of samples and d is the data dimension, is a d-dimensional set of real numbers.
[0012] Preferably, in step S2, the adjustable parameters are set The specific process of constructing the adaptive optimization correction incremental algorithm is as follows:
[0013] Entering adjustable parameters And initialize the first component parameters The formula is as follows:
[0014]
[0015] In the above formula, t represents the iteration index, T represents the total number of iterations, m represents the dimension index, n represents the data index, and d represents the data dimension. They are the mean of the initialized first component, the inverse covariance matrix, the determinant of the inverse covariance matrix, and the cumulative posterior probability;
[0016] Calculate the squared Mahalanobis distance from the data point to the existing components The formula is as follows:
[0017]
[0018] In the above formula, is the nth sample x n The square of the Mahalanobis distance to the mth component, is the mean of the mth component, is the covariance matrix of the mth component, M (t) is the number of components in the tth iteration;
[0019] The subset required for the update process is selected as follows:
[0020]
[0021] in
[0022]
[0023] In the above formula, I is the number of updates that meet the conditions, k i is the Gaussian mixture model corresponding to the i-th satisfying update condition, is the inverse of the chi-square distribution;
[0024] According to the result of Mahalanobis distance square, it is divided into update process and creation process. The classification criteria are as follows:
[0025] When the input sample Requirements: Perform the update process; if the judgment conditions are not met, perform the creation process;
[0026] The updating process is as follows: Set the tth adjustable parameter α (t) , calculate the nth sample x n In the Gaussian distribution k i The probability density of the components is:
[0027]
[0028] In the above formula, is the component k i The covariance matrix determinant of , the specific parameter update calculation is as follows:
[0029]
[0030]
[0031] In the above formula, is the kth i The posterior probability of the components is is the ratio posterior probability, is the recalculated cumulative posterior probability; is the kth i The mathematical expectation of a Gaussian mixture model, is the inverse covariance matrix after parameter update, is the inverse covariance matrix determinant;
[0032] The creation process includes: creating a new component M (t) =M (t) +1, initialize the cumulative posterior probability of the new component The formula is as follows:
[0033]
[0034] in, The new ingredients M (t) The mean, inverse covariance matrix, and determinant of the inverse covariance matrix;
[0035] The expression process of generating multiple mixed models is as follows:
[0036] In the input sample After the update process or creation process is judged, the prior probability of the component to which the sample belongs is recalculated The formula is as follows:
[0037]
[0038] At the tth iteration, according to the output Gaussian mixture model M (t) , determine the M (t) The mixture model of Gaussian components is as follows:
[0039]
[0040] In the above formula, They represent the prior probability and component parameter of the mth component respectively;
[0041] At the end of T iterations, T hybrid models are obtained. Corresponding to {α (1) ,α (2) ,…,α (T)}.
[0042] 4 Preferably, in step S3, the specific process of constructing a new regression information criterion RIC is as follows:
[0043] Determining the optimal mixture model The RIC formula is as follows:
[0044]
[0045] in, Indicates an estimate The formula for the optimal hybrid model is as follows:
[0046]
[0047] In the above formula, is the optimal mixed model.
[0048] Therefore, the present invention adopts the above-mentioned Gaussian mixture modeling method based on the adaptive optimization modified incremental algorithm, which has the following advantages:
[0049] (1) The present invention provides a Gaussian mixture modeling method based on an adaptive optimization incremental algorithm, which solves the complex problem of selecting the number of components in the existing Gaussian mixture modeling method, that is, a series of Gaussian mixture models are quickly obtained by using an adaptive adjustable parameter α and an adaptive optimization modified incremental algorithm;
[0050] (2) A Gaussian mixture modeling method based on an adaptive optimization incremental algorithm of the present invention solves the problem of difficulty in selecting the optimal model, that is, a regression information criterion RIC adaptive data sample capacity is proposed to achieve optimal model selection;
[0051] (3) The Gaussian mixture modeling method based on the adaptive optimization incremental algorithm of the present invention solves the problem of high modeling time cost, that is, as the data set increases, the optimal Gaussian mixture model can be adaptively updated, and its calculation time complexity is Reduce to
[0052] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 A schematic diagram of a Gaussian mixture modeling method based on an adaptive optimization modified incremental algorithm;
[0054] Figure 2 Flow chart of the Gaussian mixture modeling system based on adaptive optimization modified incremental algorithm;
[0055] Figure 3 It is a scatter plot of data sets in different dimensions of a Gaussian mixture modeling method based on an adaptive optimization modified incremental algorithm in an embodiment of the present invention;
[0056] Figure 4 It is a ratio diagram of BIC, AIC, SYC and RIC corresponding to the number of components M=9 as the sample size increases in the embodiment of the present invention, with 100 experiments run each time;
[0057] Figure 5 A running time comparison diagram of a Gaussian mixture modeling method based on an EM algorithm and a Gaussian mixture modeling method based on an adaptive optimization modified incremental algorithm in an embodiment of the present invention;
[0058] Figure 6 This is a two-dimensional visualization schematic diagram obtained by reducing the dimension of the MINST data set using the visualization technology t-SNE based on the adaptive optimization modified incremental algorithm of the present invention;
[0059] Figure 7is the RIC value of the criterion in a Gaussian mixture modeling method based on an adaptive optimization modified incremental algorithm in an embodiment of the present invention and the corresponding optimal Gaussian mixture model M (t) Schematic diagram of
[0060] Figure 8 The invention provides an adaptive optimization correction incremental algorithm based on the optimal number of components. Schematic diagram of contour lines. DETAILED DESCRIPTION
[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations. The specific model specifications need to be selected and determined according to the actual specifications of the device, and the specific selection calculation method adopts the existing technology in the field, so it will not be described in detail.
[0062] Example
[0063] like Figure 1 The method flow diagram shown and Figure 2 The algorithm flow chart shown in the figure, the present invention provides a Gaussian mixture modeling method based on an adaptive optimization modified incremental algorithm, comprising the following steps:
[0064] S1: Establish a Gaussian mixture model system based on the adaptive optimization modified incremental algorithm, including a data reading module, a Gaussian mixture model creation module and an adaptive optimization module;
[0065] The data reading module is responsible for reading the data set. The process is as follows: select the data in the specified file format, select As an object, where N is the number of samples and d is the data dimension, is a d-dimensional real number set;
[0066] S2: Setting adjustable parameters for Gaussian mixture model creation module And build an adaptive optimization correction incremental algorithm, combine the data set read by the data reading module, and establish a series of Gaussian mixture models;
[0067] For adjustable parameters Set the tth adjustable parameter α (t) The value range of α (t) The specific process of constructing the adaptive optimization correction incremental algorithm is as follows:
[0068] Entering adjustable parameters And initialize the first component parameters The formula is as follows:
[0069]
[0070] In the above formula, t represents the iteration index, T represents the total number of iterations, m represents the dimension index, n represents the data index, and d represents the data dimension. They are the mean of the initialized first component, the inverse covariance matrix, the determinant of the inverse covariance matrix, and the cumulative posterior probability;
[0071] Calculate the squared Mahalanobis distance from the data point to the existing components Under the formula:
[0072]
[0073] In the above formula, is the nth sample x n The square of the Mahalanobis distance to the mth component, is the mean of the mth component, is the covariance matrix of the mth component, M (t) is the number of components for the tth iteration; the subset required for the update process is selected as follows:
[0074]
[0075] in
[0076]
[0077] In the above formula, I is the number of updates that meet the conditions, k i is the Gaussian mixture model corresponding to the i-th satisfying update condition, is the inverse of the chi-square distribution;
[0078] According to the result of Mahalanobis distance square, it is divided into update process and creation process. The classification criteria are as follows:
[0079] When the input sample Requirements: Perform the update process; if the conditions are not met, perform the creation process;
[0080] The updating process is as follows: Set the tth adjustable parameter α (t) , calculate the nth sample x n In the Gaussian distribution k i The probability density of the components is:
[0081]
[0082] In the above formula, is the component ki The covariance matrix determinant of , the specific parameter update calculation is as follows:
[0083]
[0084]
[0085] In the above formula, is the kth i The posterior probability of the components is is the ratio posterior probability, is the recalculated cumulative posterior probability; is the kth i The mathematical expectation of a Gaussian mixture model, is the inverse covariance matrix after parameter update, is the determinant of the inverse covariance matrix;
[0086] The creation process includes: creating a new component M (t) =M (t) +1, initialize the cumulative posterior probability of the new component The formula is as follows:
[0087]
[0088] in, The new ingredients M (t) The mean, inverse covariance matrix, and determinant of the inverse covariance matrix;
[0089] The expression process of generating multiple mixed models is as follows: After the update process or creation process is judged, the prior probability of the component to which the sample belongs is recalculated The formula is as follows:
[0090]
[0091] At the tth iteration, according to the output Gaussian mixture model M (t) , determine the M (t) The mixture model of Gaussian components is as follows:
[0092]
[0093] In the above formula, They represent the prior probability and component parameter of the mth component respectively;
[0094] At the end of T iterations, T hybrid models are obtained. Corresponding to {α (1) ,α (2) ,…,α (T)};
[0095] S3: For the adaptive optimization module, the regression information criterion RIC of adaptive data is proposed, the size of the adaptive data set is adapted, and the optimal Gaussian mixture model in the series of Gaussian mixture models is selected.
[0096] The specific process of constructing a new regression information criterion RIC is as follows:
[0097] Determining the optimal mixture model The RIC formula is as follows:
[0098]
[0099] in, Indicates an estimate The formula for the optimal hybrid model is as follows:
[0100]
[0101] In the above formula, is the optimal mixed model.
[0102] Specific simulation experiments are carried out and evaluated in combination with real data sets, including the following two parts: efficiency evaluation and classification performance evaluation of the Gaussian mixture model (AOMI-GMM) based on the adaptive optimization modified incremental algorithm.
[0103] Part 1: Efficiency evaluation of this model
[0104] Experiments were conducted using the commonly used discriminant criteria BIC, AIC and other discriminant criteria in the establishment of Gaussian mixture models and the regression information criterion RIC proposed in this application. In order to evaluate the performance of RIC, experiments were conducted on a two-dimensional simulated data set with 9 components and different data volumes. The parameters are shown in Table 1 below:
[0105] Table 1
[0106]
[0107] The comparison results are as follows Figure 3 , is the scatter plot corresponding to the sample size [10:10:240], Figure 4 It represents that each sample is executed 100 times, M=9 proportion, and as the sample size increases, the performance comparison line chart of Akaike criterion AIC, Bayesian criterion BIC, comprehensive information criterion SYC, and RIC of the present application;
[0108] The above results show that: ① For sample sizes less than 120, the ratio of RIC corresponding to M=9 is equivalent to the ratio of AIC corresponding to M=9; ② For sample sizes greater than 120, the ratio of RIC corresponding to M=9 is between AIC and BIC. The technical solution of this application has good adaptability in both small and large sample data spaces. In order to evaluate the running time of AOMI-GMM, the calculation time is statistically analyzed, such as Figure 5 As shown in Figure 2, the time of the AOMI-GMM algorithm is much lower than that of the Gaussian mixture model based on the EM algorithm, and as the dimension d increases, the time of the EM algorithm increases quadratically.
[0109] In order to evaluate the efficiency of AOMI-GMM, the MINST handwritten digit data containing 10 grayscale handwritten digit images (from 0 to 9) is used as the input dataset. In addition, in order to gain a deeper understanding of the underlying mechanism of obtaining samples from different regions (easy to learn, blurred, and difficult to learn regions) in the sample space, the t-SNE visualization technique is used to explore the efficiency of AOMI-GMM.
[0110] The 2D visualization of the MINST dataset after dimensionality reduction using t-SNE generates the following results: Figure 6 As shown, it can be seen that there are 10 regions corresponding to the numbers 0 to 9. In the method of the present invention: the range of the logα parameter is set to [-2, -0.2], a series of RIC values and corresponding component numbers are obtained, and the minimum RIC value, that is, the optimal RIC value, is obtained. Then we can get the optimal number of components The final result verification diagram is as follows Figure 7 As shown; by specifying the uniform height level of the contour map = 0.19, 0.05 and 0.01, we get Contour plot of Graphically, the results are as follows Figure 8 The results are shown in Table 2 and compared with the standard data. Figure 6 The comparison proves the superiority of the method of the present invention.
[0111] Part II: Classification Performance Evaluation
[0112] The method of the present invention is compared with the existing methods of random forest RF, neural network NN, radial basis function support vector machine RBFSVM and SYC on six standard data sets, as shown in Table 2. At the same time, in order to avoid the accidental nature of the experimental results, this experiment uses a 10-fold cross-validation method (the training set accounts for 90% of the data, and the test set accounts for the remaining 10% of the data) for evaluation, and the process is as follows:
[0113] ① The validation data set is randomly divided into 10 parts, consisting of 10 samples of equal size from the original data set, marked as the i-th (i=1,2,...,10) sample.
[0114] ② From the 10 samples, select the i-th sample as the test sample, and the remaining samples as training samples to calculate the overall accuracy OA of the currently selected sample (expressed as OA i ), used to validate the classifier.
[0115] ③The above process is repeated 10 times by selecting different samples as test samples.
[0116] ④The final accuracy of the classifier is expressed as μ OA ±σ OA , the formula is as follows:
[0117]
[0118] The results obtained by the method of the present invention on six standard data sets are as follows, as shown in Table 2:
[0119] Table 2
[0120]
[0121]
[0122] The experimental results in Table 2 show that: 1) On average, the classification accuracy of this method is higher than that of other methods;
[0123] 2) The results show that this method is more effective than the SYC method for component determination because the coefficient of variation Smaller than the SYC method.
[0124] Therefore, the present invention adopts a Gaussian mixture modeling method based on an adaptive optimization modified incremental algorithm, adaptively generates a series of mixture models according to the adjustable parameter α, and selects the optimal number of components according to the RIC value of the mixture model. It avoids the overfitting and underfitting problems caused by manually specifying unreasonable intervals of M, and reduces the calculation. Taking multiple criteria into consideration, a new regression information criterion RIC is proposed to determine the optimal hybrid model among a series of hybrid models.
[0125] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.
Claims
1. A Gaussian mixture modeling method based on an adaptive optimization modified incremental algorithm, characterized in that: The following steps are involved: S1: Establish a Gaussian mixture model system based on the adaptive optimization modified incremental algorithm, including a data reading module, a Gaussian mixture model creation module and an adaptive optimization module; The data reading module is responsible for reading the data set; S2: Setting adjustable parameters for Gaussian mixture model creation module And build an adaptive optimization and correction incremental algorithm, combine the data set read by the data reading module, and establish a series of Gaussian mixture models; S3: For the adaptive optimization module, the regression information criterion RIC of adaptive data is proposed, the size of the adaptive data set is adapted, and the optimal Gaussian mixture model in the series of Gaussian mixture models is selected.
2. The Gaussian mixture modeling method based on the adaptive optimization modified incremental algorithm according to claim 1 is characterized in that: The process of reading a data set by the data reading module is as follows: select the data in the specified file format, select As an object, where N is the number of samples and d is the data dimension, is a d-dimensional set of real numbers.
3. The Gaussian mixture modeling method based on the adaptive optimization modified incremental algorithm according to claim 2 is characterized in that: In step S2, the adjustable parameters are set The specific process of constructing the adaptive optimization correction incremental algorithm is as follows: Entering adjustable parameters And initialize the first component parameters The formula is as follows: In the above formula, t represents the iteration index, T represents the total number of iterations, m represents the dimension index, n represents the data index, and d represents the data dimension. They are the mean of the initialized first component, the inverse covariance matrix, the determinant of the inverse covariance matrix, and the cumulative posterior probability; Calculate the squared Mahalanobis distance from the data point to the existing components The formula is as follows: In the above formula, is the nth sample x n The square of the Mahalanobis distance to the mth component, is the mean of the mth component, is the covariance matrix of the mth component, M (t) is the number of components in the tth iteration; The subset required for the update process is selected as follows: in In the above formula, I is the number of updates that meet the conditions, k i is the Gaussian mixture model corresponding to the i-th satisfying update condition, is the inverse of the chi-square distribution; According to the result of Mahalanobis distance square, it is divided into update process and creation process. The classification criteria are as follows: When the input sample Requirements: Carry out the update process; If the judgment conditions are not met, the creation process is carried out; The updating process is as follows: Set the tth adjustable parameter α (t) , calculate the nth sample x n In the Gaussian distribution k i The probability density of the components is: In the above formula, is the component k i The covariance matrix determinant of , the specific parameter update calculation is as follows: In the above formula, is the kth i The posterior probability of the components is is the ratio posterior probability, is the recalculated cumulative posterior probability; is the kth i The mathematical expectation of a Gaussian mixture model, is the inverse covariance matrix after parameter update, is the inverse covariance matrix determinant; The creation process includes: creating a new component M (t) =M (t) +1, initialize the cumulative posterior probability of the new component The formula is as follows: in, The new ingredients M (t) The mean, inverse covariance matrix, and determinant of the inverse covariance matrix; The expression process of generating multiple mixed models is as follows: In the input sample After the update process or creation process is judged, the prior probability of the component to which the sample belongs is recalculated The formula is as follows: At the tth iteration, according to the output Gaussian mixture model M (t) , determine the M (t) The mixture model of Gaussian components is as follows: In the above formula, They represent the prior probability and component parameter of the mth component respectively; At the end of T iterations, T hybrid models are obtained. Corresponding to 4. The Gaussian mixture modeling method based on the adaptive optimization modified incremental algorithm according to claim 3 is characterized in that: In step S3, the specific process of constructing a new regression information criterion RIC is as follows: Determining the optimal mixture model The RIC formula is as follows: in, Indicates an estimate The formula for the optimal hybrid model is as follows: In the above formula, is the optimal mixed model.
