GMM (Gaussian mixture model)-based chemical data analysis method

Abstract

According to the chemical data analysis method based on the Gaussian mixture model (GMM), the abnormal probability of each data point in the chemical data is accurately calculated by adopting the Gaussian mixture model (GMM), and the data change is adapted by dynamically adjusting a threshold value, so that the effective identification and segmentation of the abnormal data are realized. By analyzing the probability distribution of each data point, the initial threshold value can be set based on the abnormal probability of the data point, and it is ensured that the threshold value is closer to the distribution characteristic of actual data. According to the method and the device, automatic adjustment of the initial threshold is realized, and the threshold is updated according to the trend of the newly added data to ensure that the adjustment direction of the threshold conforms to the real-time change of the data, so that the accuracy of abnormal data judgment and the real-time performance of operation are improved. Finally, through accurate identification and adjustment, potential problems in the chemical production process are effectively monitored and analyzed, the production process is optimized, and the production safety and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a chemical data analysis method based on a Gaussian mixture model (GMM). Background Art

[0002] In order to ensure that the production quality of chemical products meets the standards, various data need to be monitored in real time during the chemical production process. These data are usually hidden in the DCS (distributed control system), covering multiple key parameters such as temperature, pressure, concentration, flow rate, and rotation speed. In complex chemical reaction processes, catalysts are often added to increase the reaction rate. However, it is not only the type and quantity of catalysts that affect chemical reactions. Temperature, pressure, raw material ratio and other conditions also play an important role. Although increasing the temperature can speed up the reaction rate, too high a temperature may cause some catalysts to fail and reduce the catalytic efficiency. Similarly, excessive pressure may also affect the reaction balance, resulting in products that do not meet expectations. In addition, slight deviations in the raw material ratio may affect the purity and yield of the product. Therefore, accurate temperature, pressure and raw material control are crucial. By analyzing the chemical data collected in the DCS system, potential abnormalities can be identified, and it can be judged whether there are production problems caused by temperature, pressure or other factors. This data-driven monitoring method provides a basis for timely adjustment of process parameters, thereby ensuring the quality of chemical products and the stability of the production process.

[0003] When using iterative threshold segmentation algorithms to identify anomalies in chemical data, the initial threshold selection is crucial because it affects the algorithm's iteration speed and segmentation results. Chemical data is real-time, and complex chemical reactions and the interaction of multiple factors may cause the data to show an upward trend. Although these trends are normal, relying solely on fixed threshold judgments can lead to misjudgments. Therefore, static thresholds are not suitable for abnormal data identification. To improve accuracy, methods that can dynamically adapt to data changes should be used. Such methods can capture multiple patterns of data and automatically adjust parameters to cope with complex operating conditions and avoid misjudging normal trends as anomalies. In addition, this method refines the judgment of data points through probability allocation, improves the robustness of decisions at fuzzy boundaries, and can adapt to real-time updates of data, thereby significantly reducing the false alarm rate.

[0004] Gaussian mixture model (GMM) is a probabilistic model that assumes that all data points are a mixture of a finite number of Gaussian distributions. It can adapt to the actual distribution of data, provide a flexible method to represent data structure, and better capture abnormal patterns in data. Summary of the invention

[0005] The present invention provides a chemical data analysis method based on Gaussian mixture model (GMM) to effectively monitor and analyze potential problems in chemical production processes.

[0006] The present invention adopts the following technical scheme:

[0007] The present invention provides a chemical data analysis method based on a Gaussian mixture model (GMM), and the method comprises the following steps: a first step, collecting data in a chemical production process; a second step, preprocessing the collected data to meet the input requirements of the Gaussian mixture model; a third step, applying the Gaussian mixture model to perform abnormal probability analysis on the preprocessed data, and calculating the abnormal probability of each data point through the model; a fourth step, determining whether the data point is abnormal according to the abnormal probability of each data point, wherein the abnormal determination is based on a preset probability threshold; and a fifth step, using the abnormal data points obtained in the fourth step to trigger response measures in the chemical production process.

[0008] Preferably, the preprocessing in the second step includes noise removal, missing value processing and data standardization, including noise removal, missing value processing and standardization processing, and the specific steps included are as follows:

[0009] Noise removal (filtering): According to the collected original data sequence {x1, x2, ..., x n}, where x i Represents the i-th data point. In order to remove noise, a moving average filter is applied to smooth the original data: Among them, x i is the data point at position i after filtering, m is half the window size, so the entire window covers 2m+1 data points. This formula means that the value of the current point is replaced by the average of the m points before and after the current point to smooth the data.

[0010] Handling missing values ​​(interpolation): There are missing values ​​(NaN) in the data. Linear interpolation is used to fill in the missing data points. The following formula is used for interpolation: i=1,2,...,k-1,where x i and x i+k is a known data point, x i+1 , x i+2 , ..., x i+k-1 is a missing value.

[0011] Standardization: In order to eliminate the influence of dimension and improve the accuracy of subsequent modeling, the data is standardized to eliminate the influence of dimension of different features. The data after denoising and interpolation is {x1′, x2′, ..., x n ′}, the standardized formula is as follows: where xi is the standardized and normalized data, μ is the mean of the data, and σ is the standard deviation of the data

[0012] Standardization converts the data into a form with zero mean and unit standard deviation, eliminating the scale differences of different features and making it suitable for subsequent modeling and analysis.

[0013] Preferably, the parameters of the Gaussian mixture model described in the third step are optimized by an expectation maximization (EM) algorithm, and the specific steps are as follows: Step E (Expectation): Calculate each data point x i The posterior probability (responsibility) of each Gaussian component: where K is the number of Gaussian components, πk is the mixing weight of the kth Gaussian component, is the probability density function of the kth Gaussian component, and its expression is defined as: μ k is the mean vector of the kth Gaussian component, ∑ k is the covariance matrix of the kth Gaussian component. d is the dimension of the data points, |∑ k | is the determinant of the covariance matrix, is the inverse of the covariance matrix.

[0014] γ ik is the calculated probability that the data point xi belongs to the kth Gaussian component. This value is called "responsibility", which indicates the probability that the Gaussian component k has on the data point x. i “Responsibility”.

[0015] M step (Maximization): Update the parameters of the Gaussian component, update the model parameters by maximizing the log-likelihood function, and use the "responsibility" in the maximization expectation step to update the parameters of the Gaussian mixture model, that is, the mean μ k , covariance matrix∑ k and the mixing weight π k .

[0016] Update the mean vector μ k ,

[0017] Update the covariance matrix,

[0018] (x i -μ k )(x i -μ k ) T is the outer product of the deviations of the data points from the mean.

[0019] Update the mixing coefficient, mixing weight π k Updated to:

[0020] Step E calculates each data point x i The posterior probability (responsibility) belonging to each Gaussian component, that is, the posterior probability, the M step uses the responsibility calculated in the E step to update the model parameters: mean, covariance matrix and mixing weights, by repeatedly iterating the E step and the M step until the model parameters converge, that is, reach the local optimum, and obtain the final estimated parameters of the GMM model.

[0021] Preferably, the abnormal probability P(x i ) is determined by comparing the actually calculated probability with a preset threshold θ, which is determined based on the performance of the model on the training data set; the specific calculation method is: Where N(x i |μ k ,∑ k ) is the data point x under the kth Gaussian component i The probability density function of P(x i )<θ, the threshold θ is optimized by minimizing the error rate on the training data set; K is the number of Gaussian components, π k is the weight of the kth Gaussian component, satisfying is the probability density function of the kth Gaussian component, whose mean is μ k , the covariance matrix is ​​∑k.

[0022]

[0023] The abnormality of a data point is associated with its probability density value in the model. The probability density value P(xi) of a data point reflects the possibility of the point appearing in the model. If the probability density value of a data point is very low, then it may be an abnormal point. Set a threshold θ to judge the abnormal point, expressed as:

[0024] if P(x i )<θ, then mark x i as anomaly.

[0026] Preferably, using the obtained abnormal data point to trigger a response measure in the chemical production process includes: when the data point x i The abnormal probability P(x i ) is less than the set threshold θ, that is, P(x i )<θ, triggering the preset response measures, and triggering an alarm when the anomaly reaches a critical level based on the severity and type of the anomaly.

[0027] The abnormal response and processing method includes the following specific steps: when abnormal data points are detected during real-time monitoring of the chemical process, a prompt response must be made to avoid potential safety hazards or production failures. The abnormal response and processing includes starting a preset response mechanism, and recording and analyzing abnormal events to ensure that effective corrective measures are taken in a timely manner.

[0028] The specific steps are as follows: 1. Start the preset response mechanism. When the system detects an abnormal data point x i When the data is marked as abnormal, the preset response mechanism is immediately activated. The response mechanism is based on the severity and type of the abnormality. When abnormal data is detected, the alarm system is immediately triggered, and a visual or auditory alarm is issued to notify the operator to pay attention.

[0029] Alarm=True ff P(x i )<θ.

[0030] 2. Record abnormal events

[0031] Each time an abnormal data point is detected, the details of the abnormal event should be recorded for subsequent analysis and tracking. The recorded information includes: abnormal data point information: such as the characteristic value of the data point, detection time, device status, etc. Detected probability density value: P(x i )<θ. Response measures taken: alarms triggered, parameters adjusted, operators notified, etc. Recording format can be structured log files or database entries for subsequent data analysis and trend tracking.

[0032] Preferably, the Gaussian mixture model (GMM) regularly updates its parameters to adapt to new data or environmental changes in the chemical process, including the following steps:

[0033] a) Incremental update: Update the model parameters π periodically or based on trigger conditions using online learning methods k , μ k and∑ k To adapt to the new data, the update formula is:

[0034]

[0035] ∑ k =β∑ k +(1-β)Cov(X new )

[0036]

[0037] b) Parameter adjustment factor: α, β, and γ are learning rate parameters that adjust the model's sensitivity to new data. k is the current mean of the kth Gaussian component, x newis the newly added sample data point, α is the learning rate parameter that controls the weight of historical data, ranging from 0 to 1. In the formula, the historical mean is given a weight of α, and the new data point is given a weight of 1-α.

[0038] Covariance matrix∑ k Update:

[0039] ∑ k =β∑ k +(1-β)Cov(X new ),

[0041] Here, ∑ k is the current covariance matrix, Cov(X new ) is the covariance contribution calculated based on the new data points, and β is the control for the influence of historical data in the update.

[0042] 3. Mixing weight π k Update:

[0043]

[0044] Among them, N k is the number of new data points attributed to the kth component, N is the total number of data points (or the number of data points in one update cycle), and γ is the learning rate parameter.

[0045] Parameter adjustment factors: α, β, and γ are learning rate parameters that adjust the model's sensitivity to new data. Higher values ​​mean that the model pays more attention to historical data, and lower values ​​make the model adapt faster.

[0046] Preferably, the method further comprises using cross validation to evaluate the parameter settings of the Gaussian mixture model to ensure the stability and accuracy of the model under different production conditions, the method comprising:

[0047] a) Data segmentation: Divide the chemical data into multiple subsets, and use the K-fold cross-validation method to alternately use some of them as test sets and the rest as training sets;

[0048] b) Performance evaluation: Calculate the performance indicators of the model on the test set after each training, such as accuracy and recall. The performance calculation formula is:

[0049] c) Parameter optimization: Optimize the parameters K and mean μ of the Gaussian mixture model based on the cross-validation results. k , and covariance∑ k , select the parameter configuration that optimizes the verification index.

[0050] Split data: Divide the chemical data into multiple subsets, and use the K-fold cross-validation method to divide the data set D into K subsets of similar size D1, D2, ..., D K Each subset serves as a test set, and the remaining K-1 subsets serve as training sets. For each cross-validation step k: training set (i.e. remove all data from the kth subset), test set (the kth subset is the test set), train a Gaussian mixture model (GMM), and Perform performance evaluation on.

[0051] Performance evaluation: Calculate the performance indicators of the model on the test set after each training, such as accuracy and recall. The accuracy calculation formula is:

[0052] Among them: TP (True Positives) is the true positive example, which indicates the number of data points correctly classified as the positive class; TN (True Negatives) is the true negative example, which indicates the number of data points correctly classified as the negative class; FP (False Positives) is the false positive example, which indicates the number of data points incorrectly classified as the positive class; FN (False Negatives) is the false negative example, which indicates the number of data points incorrectly classified as the negative class.

[0053] The recall rate focuses on how many positive samples the model can correctly identify among all positive samples. The formula is:

[0054]

[0055] Parameter optimization: Optimize the parameters K and mean μ of the Gaussian mixture model based on the cross-validation results k , covariance∑ k and the mixing weight π k , select the parameter configuration that optimizes the verification index.

[0056] Preferably, the detection of abnormal data is not limited to fixed threshold comparison, but also includes analysis of the probability density function of the data points to identify potential abnormal patterns, including:

[0057] a) Calculate the probability density of a data point: For a given data point x i , use the Gaussian mixture model to calculate its probability density function P(x i |Θ), where Θ represents the model parameter, and the calculation formula is: Where K is the number of Gaussian components, π k is the mixing weight of the kth Gaussian component, N(x i |μ k ,∑k ) is μ k is the mean, ∑ k is the probability density function of the Gaussian component of the covariance matrix;

[0058] b) Probability density threshold determination: Determine a probability density threshold δ for each data point x i , if P(x i |Θ) is lower than δ, then x i is marked as an exception, expressed as if P(x i |Θ)<δthen mark x i as anomaly, to capture complex abnormal patterns;

[0059] c) Dynamic threshold adjustment: The threshold δ is adjusted according to environmental changes and the real-time nature of the data stream. It is updated by analyzing the average probability density of the most recent N data points to ensure that the threshold reflects the latest data characteristics and changes.

[0060] The threshold is adaptively adjusted. The initial setting of the threshold Θ is based on training data or prior knowledge. In order to adapt to data changes, a dynamic adjustment mechanism is introduced. If the frequency of abnormal points exceeds expectations (indicating that the threshold may be set too high or the data has changed), Θ is adjusted to adapt to the current data environment. The adjustment strategy formula is as follows:

[0061] Θ←Θ×(1±δ),

[0063] Where δ is a factor adjusted based on the deviation of the outlier frequency relative to the expected frequency. A positive sign indicates a lower threshold, while a negative sign indicates an increase in the threshold.

[0064] The chemical data collection includes the following specific steps: placing sensors in the chemical production process, collecting data in the chemical production process through the sensors, with a collection frequency of 0.1s / time, and forming the collected data into chemical data in time sequence.

[0065] The beneficial effects of the technical solution of the present invention are as follows: the present invention adopts a Gaussian mixture model (GMM) to accurately calculate the abnormal probability of each data point in the chemical data, and dynamically adjusts the threshold to adapt to data changes, thereby realizing effective identification and segmentation of abnormal data. By analyzing the probability distribution of each data point, the present invention can not only set the initial threshold based on the abnormal probability of the data point to ensure that this threshold is closer to the distribution characteristics of the actual data, but also can efficiently and accurately distinguish abnormal data through an iterative threshold segmentation algorithm. In addition, considering that chemical data has the characteristics of real-time update, the present invention realizes automatic adjustment of the initial threshold, updates the threshold according to the trend of the newly added data, to ensure that the adjustment direction of the threshold conforms to the real-time changes of the data, thereby improving the accuracy of abnormal data judgment and the real-time nature of the operation. Finally, the present invention effectively monitors and analyzes potential problems in the chemical production process through accurate identification and adjustment, optimizes the production process, and improves production safety and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required by the embodiments or the prior art are briefly introduced below. Obviously, the drawings in the following description only represent a part of the embodiments of the present invention. For those skilled in the art, other related drawings can also be derived from these drawings without creative work.

[0067] Figure 1 It is a flow chart of the steps of the present invention. DETAILED DESCRIPTION

[0068] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of a chemical data analysis method based on big data proposed by the present invention, its specific implementation, structure, characteristics and effects, in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0069] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0070] The following is a detailed description of a specific scheme of a chemical data analysis method based on a Gaussian mixture model (GMM) provided by the present invention in conjunction with the accompanying drawings.

[0071] Chemical data collection;

[0072] Preprocess the collected data to meet the input requirements of the Gaussian mixture model, including denoising, filling missing values ​​and data standardization;

[0073] The expectation maximization (EM) algorithm is used to optimize the data to adapt to the characteristics of chemical industry, calculate the posterior probability (responsibility), and update the parameters of GMM (mean, covariance matrix, mixing weight).

[0074] The Gaussian mixture model is applied to perform anomaly probability analysis on the preprocessed data. The anomaly probability of each data point is calculated through the model. The threshold is determined based on the performance of the model training data set. The anomaly probability of the data point is compared with the threshold to make an anomaly judgment.

[0075] Model evaluation, use K-fold cross validation to evaluate model parameter settings, and optimize parameters based on the validation results. Real-time anomaly detection estimates the probability density of each new data point, sets the probability density threshold, identifies complex anomaly patterns, and adjusts the threshold based on data changes.

[0076] Dynamic threshold adjustment: adjust the anomaly detection threshold to adapt to changes in real-time operating conditions;

[0077] Abnormal event processing and response, triggering different levels of response measures according to the severity of the abnormal event, including alarm notification, automatic adjustment of control strategy or manual intervention.

[0078] like Figure 1 , which shows a flow chart of a chemical data analysis method based on big data provided by an embodiment of the present invention, the method comprising the following steps:

[0079] S001 Data acquisition: Deploy sensors to collect data from various sensors in the chemical production process, such as temperature, pressure, flow, etc.

[0080] The main purpose of the present invention is to analyze the data in the chemical production process to determine whether there is an abnormal situation. To this end, it is first necessary to collect relevant chemical data. In an embodiment of the present invention, a sensor is used to collect data in the chemical production process, and the collection frequency is once every 0.1 seconds. The collected data will constitute chemical data in chronological order. It should be pointed out that in this embodiment, the data obtained by the sensor is mainly temperature information in the chemical production process.

[0081] So far, chemical data is collected by installing sensors.

[0082] S002 performs preprocessing such as filtering and noise reduction, filling missing values, and data standardization on the collected data to meet the requirements of Gaussian mixture model input.

[0083] It should be noted that due to the heating of the sensor and other reasons, there will be a certain amount of noise in the collected chemical data, so it is necessary to first perform denoising on the obtained chemical data. In the embodiment of the present invention, denoising applies a moving average filter to smooth the original data, and the denoised data is expressed as Among them, is the data point at position i after filtering, m is half of the window size, so the entire window covers 2m+1 data points. This formula means that the value of the current point is replaced by the average of the m points before and after the current point, thereby smoothing the data.

[0084] It should be noted that the reasons for missing values ​​in chemical data collection include sensor failure, communication interruption, data transmission error, environmental interference, and equipment downtime during maintenance or calibration. These factors may cause the data to be unable to be recorded in a timely or accurate manner, leaving gaps in the data set. Dealing with these missing values ​​is crucial to ensuring the reliability of data analysis. Therefore, filling missing values ​​is a key step in data preprocessing. In an embodiment of the present invention, for missing values ​​in a sequence, linear interpolation is used to fill the missing data points, and the calculation formula is Among them, x i is the data point that needs to be filled, x i-1 and x i+1 are adjacent known data points.

[0085] It should be noted that different chemical parameters (such as temperature, pressure, concentration, etc.) may have different units and ranges. Through standardization, they can be converted to the same scale, thereby avoiding excessive influence of certain features on the model; at the same time, the standardized data can speed up the model convergence speed and improve the training effect. Therefore, standardization is an important step to ensure that the data is suitable for analysis and modeling. In the embodiment of the present invention, the standardized data Among them, x” i is the normalized data, μ and σ are the mean and standard deviation of the data respectively, and the calculation method is: Standardization converts the data into a form with zero mean and unit standard deviation, eliminating the scale differences of different features and making it suitable for subsequent modeling and analysis.

[0086] At this point, the collected chemical data has been preprocessed to ensure that the model input specification requirements are met.

[0087] S003 uses the expectation maximization (EM) algorithm to optimize the parameters of the GMM model. By repeatedly iterating the E step and the M step until the model parameters converge, that is, reach the local optimum, the final estimated parameters of the GMM model are obtained.

[0088] It should be noted that in order to effectively adapt to the characteristics of chemical data, the parameters of the Gaussian mixture model are optimized by the expectation maximization (EM) algorithm. Chemical data often have complex distributions and diverse characteristics. The EM algorithm estimates the potential structure in the data in an iterative manner, and then optimizes the model parameters, thereby improving the fitting accuracy of the data distribution, helping to identify different subgroups and outliers, and ensuring the robustness and accuracy of the model when facing actual data.

[0089] In the present invention, the parameters of the Gaussian mixture model are optimized by the expectation maximization (EM) algorithm to adapt to the characteristics of chemical data. i The posterior probability (responsibility) of each Gaussian component is given by where K is the number of Gaussian components, π k , μ k and∑ k are the weight, mean and covariance matrix of the kth component, respectively. is the probability density function of the kth Gaussian component, and its expression is defined as: M step: Update the parameters of the Gaussian component, the new mean μ k Calculated as Covariance matrix∑ k Updated to Mixing weight π k Updated to The E step calculates the "responsibility" of each data point to each Gaussian component, that is, the posterior probability. The M step uses the responsibility calculated in the E step to update the model parameters: mean, covariance matrix and mixing weights. By repeatedly iterating the E step and the M step until the model parameters converge, that is, reaching the local optimum, the final estimated parameters of the GMM model are obtained.

[0090] So far, the expectation maximization (EM) method has been used to optimize the parameters of the GMM model.

[0091] S004 obtains the probability of abnormality of each data based on the final estimated parameters of the GMM model, and compares it with the threshold of 0 to determine the abnormal point.

[0092] It should be noted that the setting of the threshold can effectively distinguish normal chemical data from abnormal chemical data. This threshold is usually determined based on the performance of the model on the training data set to ensure that it can accurately reflect the distribution characteristics of the data. In this way, the statistical information in the training set can be used to set a reasonable boundary to identify potential anomalies in practical applications. This method helps to improve the accuracy and reliability of anomaly detection and ensure safety and efficiency in the chemical process. Therefore, the abnormal probability of the chemical data point is determined by comparing the actual calculated probability with the preset threshold.

[0093] In the present example, the abnormal probability P(x i ) is determined by comparing the actual calculated probability with a preset threshold θ, which is determined based on the performance of the model on the training data set. The specific calculation method is: Where N(x i |μ k ,Σ k ) is the data point x under the kth Gaussian component i The probability density function of k .

[0094]

[0095] The abnormality of a data point is associated with its probability density value in the model. The probability density value P(x i ) reflects the probability of the point appearing in the model. If the probability density value of a data point is very low, then it may be an outlier. Set a threshold θ to judge the outlier, expressed as: if P(x i )<θ, then mark x i asanomaly.

[0096] At this point, the probability of abnormality for each data point is obtained and compared with the threshold θ to determine the abnormal point.

[0097] S005 detects abnormal data points, responds quickly to avoid potential safety hazards or production failures, initiates preset response mechanisms, and records abnormal events.

[0098] It should be noted that in chemical data analysis, the handling and response to abnormal events need to trigger different levels of measures according to their severity to ensure safety and stable production. Since abnormal events may have different degrees of impact on the environment and personnel safety, timely classification and response can effectively manage risks and give priority to high-risk situations. At the same time, by automatically adjusting control strategies and issuing alarm notifications, minor abnormalities can be quickly responded to, manual intervention can be reduced, overall efficiency and response speed can be improved, and the smoothness and safety of the production process can be ensured.

[0099] In the example of the present invention, the abnormal response and processing method includes the following specific steps:

[0100] When abnormal data points are detected during real-time monitoring of chemical processes, a prompt response is required to avoid potential safety hazards or production failures. Abnormal response and processing includes initiating a preset response mechanism, as well as recording and analyzing abnormal events to ensure that effective corrective measures are taken in a timely manner. The specific steps are as follows:

[0101] 1. Start the preset response mechanism

[0102] When the system detects an abnormal data point x i When the data is marked as abnormal, the preset response mechanism is immediately activated. The response mechanism is based on the severity and type of the abnormality. When abnormal data is detected, the alarm system is immediately triggered, and a visual or auditory alarm is issued to notify the operator to pay attention.

[0103] Alarm=True if P(x i )<θ.

[0105] 2. Record abnormal events

[0106] Each time an abnormal data point is detected, the details of the abnormal event should be recorded for subsequent analysis and tracking. The recorded information includes: abnormal data point information: such as the characteristic value of the data point, detection time, device status, etc.

[0107] Detected probability density value: P(x i )<θ. Response measures taken: alarms triggered, parameters adjusted, operators notified, etc. Recording format can be structured log files or database entries for subsequent data analysis and trend tracking.

[0108] At this point, the handling and response to abnormal events triggers different levels of measures according to their severity to ensure safety and production stability.

[0109] S006 regularly updates its parameters to adapt to new data or environmental changes in the chemical process. Online learning methods are used to update model parameters and parameter adjustment factors.

[0110] It should be noted that the data distribution of chemical processes may drift with changes in operating conditions and environmental factors. Updating model parameters can improve the accuracy of anomaly detection and classification and ensure that the model reflects the latest data features. In addition, the dynamic nature of chemical processes requires the model to be sensitive and adaptable. Regular updates help enhance the robustness of the model and avoid overfitting to old data, thereby maintaining its effectiveness and reliability in uncertain environments. Therefore, it is necessary to regularly update the parameters of the Gaussian mixture model (GMM) to adapt to new data or environmental changes in the chemical process.

[0111] In the present invention, the Gaussian mixture model (GMM) periodically updates its parameters to adapt to new data or environmental changes in the chemical process. The specific steps include: a) incremental update: periodically or based on trigger conditions, using online learning methods to update the model parameters π k , μ k and∑ k To adapt to the new data, the update formula is:

[0112]

[0113] ∑ k =β∑ k +(1-β)Cov(X new )

[0114]

[0115] Among them, μ k is the current mean of the kth Gaussian component, x new is the newly added sample data point, α is the learning rate parameter that controls the weight of historical data, ranging from 0 to 1. In the formula, the historical mean is given a weight of α, and the new data point is given a weight of 1-α. ∑ k is the current covariance matrix, Cov(X new ) is the covariance contribution calculated based on the new data point, and β controls the influence of historical data in the update. k is the number of new data points attributed to the kth component, N is the total number of data points (or the number of data points in one update cycle), and γ is the learning rate parameter.

[0116] b) Parameter adjustment factors: α, β, and γ are learning rate parameters that adjust the model’s sensitivity to new data. Higher values ​​mean that the model pays more attention to historical data, and lower values ​​make the model adapt faster.

[0117] At this point, online learning methods are used to update model parameters and parameter adjustment factors to adapt to new data or environmental changes in the chemical process.

[0118] S007 uses cross-validation to evaluate the parameter settings of the Gaussian mixture model to ensure the stability and accuracy of the model under different production conditions.

[0119] It should be noted that chemical data is usually affected by many factors, such as temperature, pressure, and raw material composition, which may cause changes in data distribution. Cross-validation can evaluate the performance of the model under different production conditions and ensure its generalization ability by dividing the data into different training and test sets. This can avoid overfitting of the model under specific conditions and improve its stability and accuracy in actual operations, thereby effectively supporting decision-making and optimizing production processes. Therefore, for chemical data, it is crucial to use cross-validation to evaluate the parameter settings of the Gaussian mixture model.

[0120] In the present invention, cross validation is used to evaluate the parameter setting of the Gaussian mixture model to ensure the stability and accuracy of the model under different production conditions. The specific method includes:

[0121] e) Data segmentation: Divide the chemical data into multiple subsets, and use the K-fold cross-validation method to alternately use a part of them as the test set and the rest as the training set.

[0122] f) Performance evaluation: Calculate the performance indicators of the model on the test set after each training, such as accuracy and recall. The performance calculation formula is: Among them: TP (True Positives) is the true positive example, which indicates the number of data points correctly classified as the positive class; TN (True Negatives) is the true negative example, which indicates the number of data points correctly classified as the negative class; FP (False Positives) is the false positive example, which indicates the number of data points incorrectly classified as the positive class; FN (False Negatives) is the false negative example, which indicates the number of data points incorrectly classified as the negative class.

[0123] g) Recall rate focuses on how many positive samples the model can correctly identify among all positive samples. The formula is:

[0124]

[0125] h) Parameter optimization: Optimize the parameters K and mean μ of the Gaussian mixture model based on the cross-validation results. k , and covariance Σ k , select the parameter configuration that optimizes the verification index.

[0126] At this point, the Gaussian mixture model is applied to calculate the abnormal probability of each data point.

[0127] S008 Detection of abnormal data: analyzing the probability density function of the data points and adaptively adjusting the threshold to identify potential abnormal patterns.

[0128] It should be noted that real-time anomaly detection can effectively identify complex anomaly patterns by estimating the probability density of each new data point and setting dynamic thresholds. Compared with the fixed threshold method, this approach enhances the adaptability and flexibility of the system, reduces the false alarm rate, and improves real-time response capabilities. By dynamically adjusting the threshold, the detection system can adapt to the changing data environment in a timely manner, thereby more accurately identifying potential risks and achieving production safety and efficient management.

[0129] In the examples of the present invention, the detection of abnormal data is not limited to fixed threshold comparison, but also includes analyzing the probability density function of the data points to identify potential abnormal patterns, specifically including:

[0130] a) Calculate the probability density of a data point: For a given data point x i , use the Gaussian mixture model to calculate its probability density function P(x i|Θ), where Θ represents the model parameter, and the calculation formula is:

[0131]

[0132] Where K is the number of Gaussian components, π k is the mixing weight of the kth Gaussian component, N(x i |μ k ,Σ k ) is μ k is the mean, Σ k is the probability density function of the Gaussian components of the covariance matrix.

[0133] b) Threshold adaptive adjustment

[0134] The initial setting of the threshold Θ is based on training data or prior knowledge. In order to adapt to data changes, a dynamic adjustment mechanism is introduced. If the frequency of abnormal points exceeds expectations (indicating that the threshold may be set too high or the data has changed), Θ is adjusted to adapt to the current data environment. The adjustment strategy formula is as follows:

[0135] Θ←Θ×(1±δ),

[0137] Where δ is a factor adjusted based on the deviation of the outlier frequency relative to the expected frequency. A positive sign indicates a lower threshold, while a negative sign indicates an increase in the threshold.

[0138] The probability density function of the data points was analyzed to identify potential abnormal patterns.

[0139] The above description is only a preferred embodiment of the present invention and does not limit the scope of the present invention. Any modification, equivalent substitution or improvement made under the principle of the present invention shall be deemed to be within the protection scope of the present invention.

Claims

1. A chemical data analysis method based on Gaussian mixture model (GMM), characterized in that: The following steps are involved: The first step is to collect data from the chemical production process; The second step is to preprocess the collected data to meet the input requirements of the Gaussian mixture model; The third step is to apply the Gaussian mixture model to perform anomaly probability analysis on the preprocessed data and calculate the anomaly probability of each data point through the model; The fourth step is to determine whether the data point is abnormal according to the abnormal probability of each data point, wherein the abnormal determination is based on a preset probability threshold; In the fifth step, the abnormal data points obtained in the fourth step are used to trigger response measures in the chemical production process.

2. A chemical data analysis method based on Gaussian mixture model (GMM) according to claim 1, characterized in that: The preprocessing described in the second step includes denoising, missing value processing and data standardization, including the following steps: Denoising: Apply a moving average filter to smooth the original data. The denoised data is expressed as Where m is half the window size, x i is the original data point; Filling missing values: For missing values ​​in the sequence, linear interpolation is used to fill the missing data points. The calculation formula is: Among them, x i is the data point that needs to be filled, x i-1 and x i+1 are adjacent known data points; Data standardization: In order to eliminate the impact of dimension and improve the accuracy of subsequent modeling, the data is standardized. Among them, x” i is the normalized data, μ and σ are the mean and standard deviation of the data respectively, and the calculation method is:

3. A chemical data analysis method based on Gaussian mixture model (GMM) according to claim 1, characterized in that: The parameters of the Gaussian mixture model described in the third step are optimized by the expectation maximization (EM) algorithm, which includes the following steps: a) Step E: Calculate each data point x i The posterior probability (responsibility) of each Gaussian component is given by where K is the number of Gaussian components, π k , μ k and Σ k are the weight, mean and covariance matrix of the kth component respectively; b) M step: Update the parameters of the Gaussian component, the new mean μ k Calculated as Covariance matrix∑ k Updated to Mixing weight π k Updated to 4. A chemical data analysis method based on Gaussian mixture model (GMM) according to claim 1, characterized in that: The abnormal probability P(x i ) is determined by comparing the actually calculated probability with a preset threshold θ, which is determined based on the performance of the model on the training data set; the specific calculation method is: Where N(x i |μ k ,∑ k ) is the data point x under the kth Gaussian component i The probability density function of The judgment is based on P(x i )<θ, and the threshold θ is optimally determined by minimizing the error rate on the training data set.

5. A chemical data analysis method based on Gaussian mixture model (GMM) according to claim 1, characterized in that: After the fourth step, the obtained abnormal data points are used to trigger response measures in the chemical production process, including: when the data point x i The abnormal probability P(x i ) is less than the set threshold θ, that is, P(x i )<θ, triggering the preset response measures, and triggering an alarm when the anomaly reaches a critical level based on the severity and type of the anomaly.

6. A chemical data analysis method based on Gaussian mixture model (GMM) according to claim 1, characterized in that: The Gaussian mixture model (GMM) periodically updates its parameters to adapt to new data or environmental changes in the chemical process, including the following steps: a) Incremental update: Update the model parameters π periodically or based on trigger conditions using online learning methods k , μ k and∑ k To adapt to the new data, the update formula is: ∑ k =βΣ k +(1-β)Cov(X new ) b) Parameter adjustment factors: α, β, and γ are learning rate parameters that adjust the model’s sensitivity to new data.

7. A chemical data analysis method based on Gaussian mixture model (GMM) according to claim 1, characterized in that: The method further includes using cross validation to evaluate parameter settings of the Gaussian mixture model to ensure the stability and accuracy of the model under different production conditions, the method comprising: a) Data segmentation: Divide the chemical data into multiple subsets, and use the K-fold cross-validation method to alternately use some of them as test sets and the rest as training sets; b) Performance evaluation: Calculate the performance indicators of the model on the test set after each training, such as accuracy and recall. The performance calculation formula is: c) Parameter optimization: Optimize the parameters K and mean μ of the Gaussian mixture model based on the cross-validation results. k , and covariance∑ k , select the parameter configuration that optimizes the verification index.

8. A chemical data analysis method based on Gaussian mixture model (GMM) according to claim 1, characterized in that: Detection of abnormal data is not limited to fixed threshold comparisons, but also includes analysis of the probability density function of data points to identify potential abnormal patterns, including: a) Calculate the probability density of a data point: For a given data point x i , use the Gaussian mixture model to calculate its probability density function P(x i |Θ), where Θ represents the model parameter, and the calculation formula is: Where K is the number of Gaussian components, π k is the mixing weight of the kth Gaussian component, N(x i |μ k ,Σ k ) is μ k is the mean, Σ k is the probability density function of the Gaussian component of the covariance matrix; b) Probability density threshold determination: Determine a probability density threshold δ for each data point x i , if P(x i |Θ) is lower than δ, then x i is marked as an exception, expressed as if P(x i |Θ)<δthen mark x i as anomaly, to capture complex abnormal patterns; c) Dynamic threshold adjustment: The threshold δ is adjusted according to environmental changes and the real-time nature of the data stream. It is updated by analyzing the average probability density of the most recent N data points to ensure that the threshold reflects the latest data characteristics and changes.

Images