Multi-factor pattern recognition method based on Gaussian mixture model

Abstract

The invention relates to a multi-factor pattern recognition method based on a Gaussian mixture model, and the method comprises the following steps: S1, carrying out the preprocessing of original data, removing incomplete feature data, and carrying out the normalization of useful features; s2, calculating the correlation degree among the features, and removing the features with high correlation degree; s3, initializing mode types, and outputting samples of each mode and dividing the samples into {C1, C2, C3,..., CN} and mean values {mu1, mu2, mu3,..., muN} of each mode; s4, parameters of a Gaussian mixture model (GMM) are initialized, and a likelihood function L (mu, sigma) and a log-likelihood function I < n > L (mu, sigma) of the GMM are obtained; s5, solving the optimal # imgabs0 # through an EM algorithm; S6, evaluating the mode recognition effect according to the solving result, and retraining the model when the evaluation effect is not met; and S7, calculating the contribution degree of each factor, and judging the influence degree of each feature on the whole data set. According to the method, through linear combination of a plurality of Gaussian distributions, complex distribution in multi-factor data of equipment can be accurately fitted, and the problem of under-fitting of a single distribution model is solved.

Description

Technical Field

[0001] The present invention belongs to the field of industrial Internet of Things, and particularly relates to a multi-factor pattern recognition method based on a Gaussian mixture model. Background Art

[0002] With the rapid development of industrial intelligence and Internet of Things technology, real-time monitoring and pattern recognition of equipment operating status have become core requirements for ensuring production efficiency and preventing failures. Current mainstream equipment monitoring technologies mostly rely on threshold alarms, traditional statistical models (such as linear regression, K-means clustering) or shallow machine learning methods. However, these methods face significant challenges in processing multi-dimensional operating data of complex equipment (such as multi-sensor signals such as vibration, temperature, and current). Equipment operating modes are often affected by multiple potential factors (such as load changes, environmental interference, and component wear), resulting in multimodal and nonlinear distribution characteristics of the data. Traditional methods are difficult to model such complex associations in actual scenarios. Equipment status is often in the transition or overlapping area between different modes. Hard classification methods (such as threshold segmentation) are prone to misjudgment, and the lack of probabilistic modeling capabilities makes it impossible to quantify uncertainty. Summary of the Invention

[0003] To address these issues, this paper proposes a multi-factor pattern recognition method based on the Gaussian mixture model (GMM). By fusing multi-source sensor data and leveraging the probabilistic generation and soft clustering capabilities of the GMM, it achieves high-precision, adaptive modeling and identification of equipment operating status. This technology overcomes the limitations of traditional methods and provides an innovative solution for industrial equipment health management.

[0004] To solve the above technical problems, the present invention provides a multi-factor pattern recognition method based on a Gaussian mixture model, comprising the following steps:

[0005] S1. Preprocessing of raw data: removing incomplete feature data and normalizing useful features;

[0006] S2. Calculate the correlation between each feature and remove features with high correlation;

[0007] S3, initialize the mode type, output each mode sample divided into {C1, C2, C3, ..., C N} and the mean of each mode {μ1,μ2,μ3,...,μ N};

[0008] S4. Initialize the Gaussian mixture model (GMM) parameters and obtain the GMM likelihood function L(μ,Σ) and log-likelihood function ΙnL(μ,Σ);

[0009] S5, EM algorithm to find the optimal solution

[0010] S6. Evaluate the pattern recognition effect based on the solution results. If the evaluation effect is not satisfactory, the model needs to be retrained.

[0011] S7. Calculate the contribution of each factor and determine the impact of each feature on the overall data set.

[0012] Preferably, in step S1, the raw data preprocessing is divided into the following steps:

[0013] The time series data set X obtained from IoT devices is: {x1,x2,...,x m}, m represents the number of data sets, for each data set x i , there are n features {a1,a2,...,a n}, the time series dataset is represented as:

[0014]

[0015] Step 1: Remove the eigenvalues with high data loss rate. For feature a i , assuming that the number of lost data is j, and set the threshold σ (data integrity), then when When , feature a is retained i , otherwise it is removed. At this time, the number of features is l, and the time series data set is expressed as:

[0016]

[0017] Step 2: Fill in the missing data and take the mean of the existing data Replace the missing data. When calculating, the missing data will be treated as 0. The calculation formula is:

[0018]

[0019] Step 3: Normalize different eigenvalues. i , each a it Normalized value a' it Expressed as:

[0020]

[0021] Among them, min(a i )、max(a i ) are feature a i The minimum and maximum values of

[0022] Step 4: Obtain the normalized time series data matrix;

[0023]

[0024] Preferably, in step S2, calculating the correlation between the features and eliminating the features with high correlation is divided into the following steps:

[0025] Step 1: Calculate the mean of each feature. For feature a i , its mean Expressed as:

[0026]

[0027] Step 2: Calculate the standard deviation of each feature. i , its standard deviation Expressed as:

[0028]

[0029] Step 3: Calculate the covariance between any two features. For feature a i 、a h , its standard deviation cov(a i ,a h ) is expressed as:

[0030]

[0031] Step 4: Calculate the correlation between any two features. For feature a i 、a h , the correlation degree r(a i ,a h ) is expressed as:

[0032]

[0033] Step 5: Feature screening, set the correlation threshold s, which is a number between (0, 1). The closer it is to 1, the higher the correlation between the two features. When it is 0, it means that the two features are completely unrelated. When it is 1, it means that the two features are the same feature. For feature a i 、a h , select as follows:

[0034]

[0035] Preferably, the initialization mode type in step S3 is divided into the following steps:

[0036] The first step is to select one piece of data from each of N working modes from the sample set as the mean of the N modes;

[0037] The second step is to calculate each sample x i The distance d from the mean of all modes in ;

[0038]

[0039] Among them, n represents the nth working mode, μ n Represents the mean of the current nth working mode, and then each sample x i Assign to the closest mode C n In the formula, it is expressed as follows:

[0040]

[0041] The third step is to update the mean value of each mode. The calculation formula is as follows:

[0042]

[0043] Among them, |C n | indicates mode C n The number of samples.

[0044] Step 4: Repeat steps 2 and 3 until the mean of each mode no longer changes, and finally output the samples of each mode divided into {C1, C2, C3, ..., C N} and the mean of each mode {μ1,μ2,μ3,...,μ N}.

[0045] Preferably, the Gaussian mixture model (GMM) parameter initialization in step S4 is divided into the following steps:

[0046] In the first step, the pattern classification results of step 3 are used as the initial mean of each Gaussian distribution of GMM;

[0047] The second step is to calculate the covariance matrix Σ under each mode K The calculation formula is as follows:

[0048]

[0049] The third step is to calculate the initial weights of GMM {π1,π2,π3,...,π N}; All weights must satisfy π k The calculation formula is as follows:

[0050]

[0051] The fourth step is to obtain the probability density function P(x|μ,Σ) of the Gaussian mixture model;

[0052]

[0053] Among them, p(x i |μ n ,Σ n ) represents the probability density function of the nth mode, and the calculation formula is:

[0054]

[0055] The fifth step is to obtain the likelihood function L(μ,Σ) and log-likelihood function ΙnL(μ,Σ) of GMM.

[0056]

[0057] Preferably, the EM algorithm in step S5 solves the optimal It is divided into the following steps:

[0058] The first step is to calculate the Gaussian component posterior responsibility γ(z ik );

[0059]

[0060] The second step is to calculate the posterior responsibility γ(z ik )renew

[0061]

[0062] In the third step, the first and second steps are iterated continuously until the incremental threshold of the log-likelihood is less than the range ε. The value of ε is determined according to the actual situation:

[0063] |ΙnL(μ,Σ) (t+1) -ΙnL(μ,Σ) t |<ε.

[0064] Preferably, the pattern recognition effect evaluation in step S6 is divided into the following steps:

[0065] The first step is to calculate the variance of the weight Var(π) to check whether the weight is reasonably distributed;

[0066]

[0067] When Var(π) is close to 0 or very large, repeat steps 3, 4, and 5;

[0068] The second step is to calculate the recognition of the new sample, take the test data of S quantity for category analysis, and calculate x s and The distance d sk , and find the two with the smallest distance and The difference The calculation formula is:

[0069]

[0070] Set the distance threshold D ε And statistics Less than the distance threshold D ε The number num, when When the number of statistics increases by 1; finally, according to the ratio threshold R ε Determine whether the sample recognition meets the requirements. Repeat steps 3, 4, and 5.

[0071] The third step is to calculate the fitness of the new sample to the model, based on the minimum distance obtained in the second step Determine that the sample belongs to category k and calculate the posterior probability P of the sample observation value. If the posterior probability of category k is less than Repeat steps 3, 4, and 5.

[0072] Preferably, the formula for calculating the contribution of each factor in step S7 is:

[0073]

[0074] Among them, i represents the characteristics of the i-th factor, μ k,i It represents the i-th factor in the k-th Gaussian component. The greater the difference in the mean of the factor between different modes and the more significant the difference in the high-weight component, the higher the contribution.

[0075] Compared with the prior art, the present invention has the following beneficial effects:

[0076] 1. High-precision multimodal modeling: The present invention can accurately fit the complex distributions (such as asymmetric and multimodal distributions) in the multi-factor data of the device through the linear combination of multiple Gaussian distributions, overcoming the underfitting problem of a single distribution model.

[0077] 2. Dynamic adaptive learning: The present invention combines the online EM algorithm or incremental learning strategy, and the model can update the mixing weights and sub-distribution parameters in real time to adapt to the drift of equipment operating conditions.

[0078] 3. Explainability and interventionability: This invention determines the number of classifications through the existing control mode, ensuring that the final classification results are consistent with the actual scenario. On the other hand, it reduces the probability of overfitting and underfitting of the model through manual intervention.

[0079] 4. Improved computing efficiency and implementation feasibility: This invention improves computing efficiency through raw data preprocessing and feature dimensionality reduction, reduces the number of subsequent model parameters, and can achieve parallel modeling of large-scale equipment to meet the real-time requirements of the Internet of Things industrial scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] The present invention will be further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0081] Figure 1 It is a structural schematic diagram of the present invention. DETAILED DESCRIPTION

[0082] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0083] like Figure 1 According to one aspect of the present invention, a multi-factor pattern recognition method based on a Gaussian mixture model is provided, comprising the following steps:

[0084] Step 1: Preprocess the raw data to remove incomplete feature data and normalize useful features;

[0085] The time series data set X obtained from IoT devices is: {x1,x2,...,x m}, m represents the number of data sets. In this example, m is set to 1000 (for the convenience of calculation, a larger value will be selected based on the actual computing resources). For each data set x i , there are n features {a1,a2,...,a n In this example, we take the five features of the device: {voltage, current, power, temperature, and energy consumption}. The time series dataset is represented as follows:

[0086]

[0087] Step 1: Remove the eigenvalues with high data loss rate. For feature a i , assuming that the number of lost data is j, and set the threshold σ (data integrity), in this example σ is 0.8, then when When , feature a is retained i Otherwise, it is removed. Assuming that the energy consumption data is incomplete, the features are {voltage, current, power, temperature}, and the time series data set is represented as:

[0088]

[0089] Step 2: Fill in the missing data and take the mean of the existing data Replace the missing data. When calculating, the missing data will be treated as 0. The calculation formula is expressed as:

[0090]

[0091] Step 3: Normalize different eigenvalues. i , each a it Normalized value a' it Expressed as:

[0092]

[0093] Among them, min(a i )、max(a i ) are feature a i The minimum and maximum values of .

[0094] Step 4: Obtain the normalized time series data matrix;

[0095]

[0096] Step 2: Calculate the correlation between each feature and remove features with high correlation.

[0097] Step 1: Calculate the mean of each feature. For feature a i , its mean Expressed as:

[0098]

[0099] Step 2: Calculate the standard deviation of each feature. i , its standard deviation Expressed as:

[0100]

[0101] Step 3: Calculate the covariance between any two features. For feature a i 、a h , its standard deviation cov(a i ,a h ) is expressed as:

[0102]

[0103] Step 4: Calculate the correlation between any two features. For feature a i 、a h , the correlation degree r(a i ,a h ) is expressed as:

[0104]

[0105] Step 5: Feature screening, set the correlation threshold s, which is a number between (0, 1). The closer it is to 1, the higher the correlation between the two features. When it is 0, it means that the two features are completely unrelated, and when it is 1, it means that the two features are the same feature. In this example, s is set to 0.7 (select according to the actual situation. When there are many features, s can be appropriately smaller to reduce the subsequent solution dimension). For feature a i 、a h , select as follows:

[0106]

[0107] Step 3: Initialize the mode type. Assume that the IoT device has N operating modes. In this example, assume that the device has five modes: {manual, automatic, overload, low voltage, and alarm}. In this case, N is 5.

[0108] The first step is to select one piece of data for each of the five working modes from the sample set as the mean of the five modes.

[0109] The second step is to calculate each sample x i The distance d from the mean of all modes in .

[0110]

[0111] Among them, n represents the nth working mode, μ n Represents the mean of the current nth working mode, and then each sample x i Assign to the closest mode C n middle.

[0112]

[0113] The third step is to update the mean value of each mode. The calculation formula is as follows:

[0114]

[0115] Among them, |C n | indicates mode C n The number of samples.

[0116] In the fourth step, the second and third steps are repeated until the mean of each mode no longer changes. Finally, the samples of each mode are divided into {C1, C2, C3, C4, C5} and the mean of each mode {μ1, μ2, μ3, μ4, μ5} are output.

[0117] Step 4: Initialize Gaussian mixture model (GMM) parameters.

[0118] In the first step, the pattern classification result of step 3 is used as the initial mean of each Gaussian distribution of GMM.

[0119] The second step is to calculate the covariance matrix Σ under each mode K , the calculation formula is as follows:

[0120]

[0121] The third step is to calculate the initial GMM weights {π1,π2,π3,π4,π5}, and all weights must satisfy: π k The calculation formula is as follows:

[0122]

[0123] The fourth step is to obtain the probability density function P(x|μ,Σ) of the Gaussian mixture model;

[0124]

[0125] Among them, p(x i |μ n ,Σ n ) represents the probability density function of the nth mode, and the calculation formula is:

[0126]

[0127] Step 5: Obtain the likelihood function L(μ,Σ) and log-likelihood function ΙnL(μ,Σ) of GMM;

[0128]

[0129] Step 5: EM algorithm finds the optimal solution

[0130] The first step is to calculate the Gaussian component posterior responsibility γ(z ik );

[0131]

[0132] The second step is to calculate the posterior responsibility γ(z ik )renew

[0133]

[0134] In the third step, the first and second steps are iterated continuously until the incremental threshold of the log-likelihood is less than the range ε. The value of ε is determined according to the actual situation. In this example, the value of ε is 10 -6 .

[0135] |ΙnL(μ,Σ)(t+1) -ΙnL(μ,Σ) t |<10 -6

[0136] Step 6: Pattern recognition effect evaluation.

[0137] The first step is to calculate the variance of the weight Var(π) to check whether the weight is reasonably distributed.

[0138]

[0139] When Var(π) is close to 0 or very large, repeat steps 3, 4, and 5;

[0140] The second step is to calculate the recognition of the new sample. Take S test data for category analysis. In this example, 4 test data are taken for each of the 5 operating modes, totaling 20 test data, and calculate x s and The distance d sk , and find the two with the smallest distance and The difference The calculation formula is:

[0141]

[0142] Set the distance threshold D ε And statistics Less than the distance threshold D ε The number num,

[0143] when When the number of statistics increases by 1; finally, according to the ratio threshold R ε Determine whether the sample recognition meets the requirements. Repeat steps 3, 4, and 5.

[0144] The third step is to calculate the fitness of the new sample to the model. According to the minimum distance obtained in the second step Determine that the sample belongs to category k and calculate the posterior probability P of the sample observation value. If the posterior probability of category k is less than Repeat steps 3, 4, and 5.

[0145] Step 7: Calculate the contribution of each factor. The difference in the mean of factors in different Gaussian components (modes) combined with the component weights can directly reflect the factor discrimination ability. The formula is:

[0146]

[0147] Among them, i represents the i-th factor (feature), μ k,iIt represents the mean of the i-th factor in the k-th Gaussian component. The greater the difference in the mean of the factor between different modes and the more significant the difference in the high-weight components, the higher the contribution.

[0148] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-factor pattern recognition method based on Gaussian mixture model, characterized in that: The following steps are involved: S1. Preprocessing of raw data: removing incomplete feature data and normalizing useful features; S2. Calculate the correlation between each feature and remove features with high correlation; S3, initialize the mode type, output each mode sample divided into {C1, C2, C3, ..., C N } and the mean of each mode {μ1,μ2,μ3,...,μ N }; S4. Initialize the parameters of the Gaussian mixture model GMM and obtain the likelihood function L(μ,Σ) and log-likelihood function ΙnL(μ,Σ); S5, EM algorithm to find the optimal solution S6. Evaluate the pattern recognition effect based on the solution results. If the evaluation effect is not satisfactory, the model needs to be retrained. S7. Calculate the contribution of each factor and determine the impact of each feature on the overall data set.

2. A multi-factor pattern recognition method based on Gaussian mixture model according to claim 1, characterized in that: In step S1, the raw data preprocessing is divided into the following steps: The time series data set X obtained from IoT devices is: {x1,x2,...,x m }, m represents the number of data sets, for each data set x i , there are n features {a1,a2,...,a n }, the time series dataset is represented as: Step 1: Remove the eigenvalues with high data loss rate. For feature a i , assuming that the number of lost data is j, and set the threshold σ, then when When , feature a is retained i , otherwise it is removed. At this time, the number of features is l, and the time series data set is expressed as: Step 2: Fill in the missing data and take the mean of the existing data Replace the missing data. When calculating, the missing data will be treated as 0. The calculation formula is: Step 3: Normalize different eigenvalues. i , each a it Normalized value a' it Expressed as: Among them, min(a i )、max(a i ) are feature a i The minimum and maximum values of Step 4: Obtain the normalized time series data matrix; 3. A multi-factor pattern recognition method based on Gaussian mixture model according to claim 2, characterized in that: In step S2, the correlation between the features is calculated, and the features with high correlation are eliminated in the following steps: Step 1: Calculate the mean of each feature. For feature a i , whose mean μ' ai Expressed as: Step 2: Calculate the standard deviation of each feature. i , its standard deviation σ ai Expressed as: Step 3: Calculate the covariance between any two features. For feature a i 、a h , its standard deviation cov(a i ,a h ) is expressed as: Step 4: Calculate the correlation between any two features. For feature a i 、a h , the correlation degree r(a i ,a h ) is expressed as: Step 5: Feature screening, set the correlation threshold s, which is a number between (0, 1). The closer it is to 1, the higher the correlation between the two features. When it is 0, it means that the two features are completely unrelated. When it is 1, it means that the two features are the same feature. For feature a i 、a h , select as follows:

4. A multi-factor pattern recognition method based on Gaussian mixture model according to claim 3, characterized in that: The initialization mode type in step S3 is divided into the following steps: The first step is to select one piece of data from each of N working modes from the sample set as the mean of the N modes; The second step is to calculate each sample x i The distance d from the mean of all modes in ; Among them, n represents the nth working mode, μ n Represents the mean of the current nth working mode, and then each sample x i Assign to the closest mode C n In the formula, it is expressed as follows: The third step is to update the mean value of each mode. The calculation formula is as follows: Among them, |C n | indicates mode C n The number of samples; Step 4: Repeat steps 2 and 3 until the mean of each mode no longer changes, and finally output the samples of each mode divided into {C1, C2, C3, ..., C N } and the mean of each mode {μ1,μ2,μ3,...,μ N }.

5. The multi-factor pattern recognition method based on Gaussian mixture model according to claim 4, characterized in that: The initialization of the Gaussian mixture model GMM parameters in step S4 is divided into the following steps: In the first step, the pattern classification results of step 3 are used as the initial mean of each Gaussian distribution of GMM; The second step is to calculate the covariance matrix Σ under each mode K The calculation formula is as follows: The third step is to calculate the initial weights of GMM {π1,π2,π3,...,π N }; All weights must satisfy π k The calculation formula is as follows: The fourth step is to obtain the probability density function P(x|μ,Σ) of the Gaussian mixture model; Among them, p(x i |μ n ,Σ n ) represents the probability density function of the nth mode, and the calculation formula is: Step 5: Obtain the likelihood function L(μ,Σ) and log-likelihood function ΙnL(μ,Σ) of GMM; 6. The multi-factor pattern recognition method based on Gaussian mixture model according to claim 5, characterized in that: In step S5, the EM algorithm solves the optimal It is divided into the following steps: The first step is to calculate the Gaussian component posterior responsibility γ(z ik ); The second step is to calculate the posterior responsibility γ(z ik )renew In the third step, the first and second steps are iterated continuously until the incremental threshold of the log-likelihood is less than the range ε. The value of ε is determined according to the actual situation: |ΙnL(μ,Σ) (t+1) -ΙnL(μ,Σ) t |<e。 7. The multi-factor pattern recognition method based on Gaussian mixture model according to claim 1, characterized in that: The pattern recognition effect evaluation in step S6 is divided into the following steps: The first step is to calculate the variance of the weight Var(π) to check whether the weight is reasonably distributed; When Var(π) is close to 0 or very large, repeat steps 3, 4, and 5; The second step is to calculate the recognition of the new sample, take the test data of S quantity for category analysis, and calculate x s and The distance d sk , and find the two with the smallest distance and The difference The calculation formula is: Set the distance threshold D ε And statistics Less than the distance threshold D ε The number num, when When the number of statistics increases by 1; finally, according to the ratio threshold R ε Determine whether the sample recognition meets the requirements. Repeat steps 3, 4, and 5. The third step is to calculate the fitness of the new sample to the model, based on the minimum distance obtained in the second step Determine that the sample belongs to category k and calculate the posterior probability P of the sample observation value. If the posterior probability of category k is less than Repeat steps 3, 4, and 5.

8. The multi-factor pattern recognition method based on Gaussian mixture model according to claim 1, characterized in that: The formula for calculating the contribution of each factor in step S7 is: Among them, i represents the characteristics of the i-th factor, μ k,i It represents the i-th factor in the k-th Gaussian component. The greater the difference in the mean of the factor between different modes and the more significant the difference in the high-weight component, the higher the contribution.

Images