Method for constructing non-parametric EWMA control chart based on conformal P value

Abstract

The invention belongs to the technical field of statistical process control, and discloses a method for constructing a non-parametric EWMA control chart based on a conformal P value. The method comprises the following steps: before establishing a control chart, acquiring ion concentration data with unknown distribution and unknown dimensions; when the statistical magnitude of the control chart is established, firstly calculating a conformal P value of each sample, and then aggregating the calculated P values by using the idea of Z test to construct a test statistical magnitude; a Monte Carlo simulation mode is used to generate a large amount of data obeying standard normal distribution, a conformal P value of each piece of ion concentration data to be monitored is calculated, a corresponding statistical magnitude is established to approximately obtain empirical distribution of the statistical magnitude of a control chart in a controlled state, and then an alpha quantile of the distribution is taken as a final control line; when the statistical magnitude of the control chart is smaller than the control line, the ion concentration in the process is not within the controlled range; otherwise, the ion concentration in the process is in a controlled range.

Description

Technical Field

[0001] The present invention relates to the technical field of statistical process control, and in particular to a method for constructing a non-parametric EWMA control chart based on a conformal P value, wherein the improvement mainly applies the conformal P value to construct a non-parametric EWMA control chart, i.e., an EWMA control chart without distribution. Background Art

[0002] In the production process of hydrometallurgy, the application of statistical process control (SPC) is crucial. As a commonly used visualization tool in SPC, the control chart can continuously monitor the ion concentration in the solution, which is a key factor in ensuring the stable operation of the entire hydrometallurgical process and the compliance of product quality with standards. By establishing a control chart, operators can obtain real-time ion concentration data, which reflects the concentration level of metal ions in the solution and is essential for controlling metal extraction efficiency and purity. When these data are collected and analyzed in real time, any deviation from the predetermined control range can be quickly identified. When the monitored ion concentration exceeds the preset control limit, the system will automatically issue an alarm to remind the operator to check and handle it immediately. This timely feedback mechanism is crucial to preventing potential quality problems because it can prevent small deviations from turning into major production accidents, reduce resource waste, and improve product consistency and reliability.

[0003] Common control charts mainly include Shewhart control charts, CUSUM control charts and EWMA control charts. However, these three control charts are all based on the normal assumption. However, in practical applications, not all data obey the normal distribution. At this time, if the control charts designed under the normal assumption are continued to be used, these distribution assumptions will be unreasonable and the distribution will be incorrectly specified, which will have a potential impact on the performance of the control chart. Therefore, in this case, it is very necessary to design a distribution-free or non-parametric control chart.

[0004] Existing distribution-free or nonparametric control charts can quickly detect drift, but when the distribution of process data is unknown, sufficient reference samples are required to achieve the desired monitoring effect. Insufficient reference samples prevent us from using resampling methods or distribution function estimation to find control lines. Most existing nonparametric control charts are proposed for one-dimensional variables and multidimensional variables separately. If the components of multidimensional data are independent, then multiple univariate distribution-free control charts can be considered, however, the unknown correlation structure between components makes this method infeasible. Summary of the invention

[0005] In order to overcome the above technical defects, the present invention proposes a method for constructing a nonparametric EWMA control chart based on conformal P value, and the proposed control chart can adapt to data of various distribution types and dimensions, and can implement efficient monitoring of the process even when the reference sample size is limited. The specific technical solution is as follows:

[0006] S1. Acquisition of data required for establishing control charts: Before establishing control charts, first obtain ion concentration data with unknown distribution and unknown dimension. The process data can be normal distribution, t distribution, chi-square distribution or mixed distribution data;

[0007] S2. Determination of the statistics of the control chart: When establishing the statistics of the control chart, the conformal P value of each ion concentration data must be calculated first, and then the calculated P values ​​are aggregated using the idea of ​​the Z test to construct the test statistic. The specific implementation steps are as follows:

[0008] S2.1. First, obtain the historical process ion concentration data and the ion concentration data set of the process to be monitored whose distribution obeys any unknown distribution. Divide the historical data into two sets, namely the training set and the calibration set. Use the data in the training set to train the scoring function; apply the trained scoring function to the calibration data set and the ion concentration data to be monitored to obtain the scores of the calibration set and the process data to be monitored. The score is the abnormality score of the sample. Generally, the higher the score, the more abnormal the ion concentration.

[0009] S2.2. Apply the calculated score to the formula for calculating the standard conformal P value, and you can get the conformal P value corresponding to the ion concentration data at each moment to be monitored, as well as the conformal P value of the (ω-1) ion concentration data in the calibration set.

[0010] S2.3. The statistic at the first moment to be monitored is: aggregate the conformal P value corresponding to the first moment to be monitored and the conformal P value of (ω-1) samples in the calibration set, and use the idea of ​​Z test to construct the statistics of the control chart; the statistic at the second moment to be monitored is: aggregate the conformal P value corresponding to the ion concentration data at the second moment to be monitored, the conformal P value corresponding to the ion concentration data at the first moment to be monitored, and the conformal P value of (ω-2) ion concentration samples in the calibration set, and use the idea of ​​Z test to construct the statistics of the control chart.

[0011] S2.4, the choice of ω is related to the smoothing parameter λ, and the selected ω satisfies the formula (1-λ) ω ≈0 is enough.

[0012] S3. Determine the control line: Use the Monte Carlo simulation method to first generate a large amount of data that obeys the standard normal distribution. Part of this data is used as the reference sample n0, and part is used as the test sample m. Calculate the conformal P value of each test sample, and establish the corresponding statistic according to the method in S2. Repeat this process multiple times to approximate the empirical distribution of the statistics of the control chart under the controlled state, and then take the quantile of the distribution as the control line corresponding to the m test samples, and take the maximum value of all control line values ​​as the final control line. The control lines with the same reference sample size and parameters can be universal, and the target operation chain length ARL0=1 / α under the controlled state;

[0013] S4. Determine whether the process is under control: Since the data generated by abnormal ion concentration has a smaller P value, the statistic will also tend to a smaller value when the ion concentration is abnormal. Therefore, when the statistic of the control chart is less than the control line, the control chart will issue an alarm, indicating that the ion concentration in the process is out of control; otherwise, it means that the ion concentration in the process is under control.

[0014] The technical solution of the present invention has the following beneficial effects:

[0015] This paper applies conformal inference theory to the SPC process, providing a new idea for constructing nonparametric control charts. The idea of ​​Z test is used to construct test statistics. In addition, a time window is introduced, which not only reduces the computational burden but also effectively filters the impact of historical observations on current observations. The establishment of the control line of the control chart proposed in this paper does not depend on the distribution of observed data or the score function, but only on the previously established false alarm probability α, smoothing parameter λ and reference sample size n. Thanks to the flexibility of the black box algorithm, the constructed control chart can implement efficient monitoring of the ion concentration control process without considering the data distribution and dimension. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flow chart of a method for constructing a nonparametric EWMA control chart based on a conformal P value in the present invention;

[0017] Figure 2 The scoring functions that can be used to construct the proposed control charts;

[0018] Figure 3 is the control line under different target average operation chain lengths, different numbers of historical controlled data n0, and different parameters λ;

[0019] Figure 4 The experimental results of Example 1 show the average running chain length performance of the control chart under different mean shifts compared with the NLE control chart;

[0020] Figure 5 The experimental results of Example 2 show the average running chain length performance of the control chart under different variance drifts compared with the NLE control chart. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0022] In the description of the present invention, it should be understood that the numerical labels before the steps do not determine the order of the steps, but are only used to facilitate the description of the present invention and distinguish each step, and therefore cannot be understood as a limitation of the present invention.

[0023] First, the following terms are explained:

[0024] Monte Carlo simulation: Monte Carlo simulation is a numerical computing technique that uses random sampling and probability statistics to solve complex problems. The basic idea is to simulate the behavior of the system through a large number of random experiments to estimate the solution to the problem. Monte Carlo simulation can estimate the expected value or probability distribution by simulating many different possible scenarios. Its main advantage is that it can handle high-dimensional, nonlinear and complex uncertain systems, but its disadvantage is that it requires a lot of computing resources, and the results are approximate values, and the accuracy depends on the number of samples and the sampling method. With the improvement of computing power, Monte Carlo simulation has become an important tool for solving many practical problems.

[0025] Control Chart is a method used to monitor and analyze process stability and quality control. It is widely used in industrial engineering, production management and quality control.

[0026] The statistics of a control chart refer to the numerical values ​​used to describe data characteristics and process status in the control chart. The statistics of a control chart are used to determine whether the process is in a "controlled state" (i.e., the fluctuation of the process is within a predetermined range) or an "out-of-control state" (i.e., the process has abnormal fluctuations and measures need to be taken) by monitoring the key characteristics of the process (such as mean, range, standard deviation, etc.).

[0027] The EWMA (Exponentially Weighted Moving-Average) control chart is an exponentially weighted moving average control chart, which is mainly used to monitor the mean change of the process and is a statistical process control tool. It performs weighted processing on historical data. The closer the data is to the current data, the greater the weight is, and the small deviations in the process can be monitored more timely.

[0028] As described in the background, in view of the defects of the existing non-parametric control charts, the present invention designs a non-parametric EWMA control chart based on conformal P value to compensate for the defects of the existing control charts and monitor the process. In order to facilitate those skilled in the art to understand the technical solutions provided in the embodiments of the present invention, the relevant technologies are described below. Figure 1 As shown, the method for constructing a nonparametric EWMA control chart based on a conformal P value in an embodiment of the present invention includes:

[0029] S1. Acquisition of ion concentration data required for establishing control chart:

[0030] Before establishing a control chart, first obtain ion concentration data with unknown distribution and unknown dimension.

[0031] Specifically, the ion concentration data of the historical controlled process and the ion concentration data of the process to be monitored are obtained, and the distributions of the two sets of data are both unknown distributions.

[0032] If the ion concentration of the process to be monitored is in a controlled state, then the ion concentration data of the process to be monitored should have the same unknown distribution as the ion concentration data of the historical controlled process, denoted as F0. If the process to be monitored is out of control, then the ion concentration data of the process to be monitored should obey another unknown distribution, denoted as F1. In other words, when the process to be monitored is in a controlled state, the ion concentration data of the process to be monitored has the same unknown distribution F0 as the ion concentration data of the historical controlled process. When the process to be monitored is out of control, the ion concentration data of the process to be monitored obeys another unknown distribution F1.

[0033] First, obtain the ion concentration data Y of the historical controlled process i , there are n0 ion concentration data of historical controlled processes, and the ion concentration data set of this historical controlled process is represented by D. The historical controlled ion concentration data obeys an unknown distribution F0. And the ion concentration data X of the process to be monitored generated over time t , t=1,2,…, the ion concentration data of the process to be monitored X t It is possible that over time, the distribution F0 changes to the distribution F1, where F1 is an unknown distribution different from F0.

[0034] Specifically, the present invention uses the drift of data mean and variance to reflect the change of data distribution, using δ μ and δ σ Represents the drift of mean and variance. If δ μ =0,δ σ =0 means the data to be monitored X t It obeys the distribution F0, that is, until time t, the monitored process is in a controlled state. μ ≠0,δ σ ≠0 means the ion concentration data X of the process to be monitored t Subject to the distribution F1, the monitored process is out of control. The time when the data of the monitored process drifts is recorded as τ, and τ=0.

[0035] S2. Determination of statistics of control chart:

[0036] When establishing the statistics of the control chart, we must first calculate the conformal P value of each sample, and then use the idea of ​​the Z test to aggregate the calculated P values ​​to construct the test statistic.

[0037] The calculation of statistics is as follows: First, calculate the conformal P value corresponding to each ion concentration data of the process to be monitored (that is, each data in the process data to be monitored), as well as the conformal P value of (ω-1) data in the historical controlled process ion concentration data. Where ω is the time window, which enables only the ω ion concentration data closest to the current moment (including the current measured data itself) to be considered each time when calculating the statistics. The purpose of calculating the conformal P value of the ion concentration data of (ω-1) historical controlled processes is to assist in establishing the statistics of the control chart to enable monitoring of the process.

[0038] For the calculation of conformal P value, the ion concentration data set D of the historical controlled process is first divided into the calibration set D cal and training set D tr , using the training set D tr A scoring function is trained based on the ion concentration data in the dataset, which can quantify the abnormality of the data. The proposed classification model is shown in the attached figure. Figure 2 Given in Figure 2 The score functions shown include the average distance of K nearest neighbors, the average path length, the distance to the hyperplane, the distance to the mean of the training set, the distance to the median of the training set, and the Mahalanobis distance. In actual operation, it is not limited to these score functions. Use the trained model to analyze the ion concentration data in the calibration set and the ion concentration data of the process to be monitored X t An abnormality score is performed, and the abnormality score is recorded as S(·). The larger the score is, the more abnormal the corresponding ion concentration data may be.

[0039] Based on the calculated abnormality score, the conformal P value p of the ion concentration data of the process to be monitored is calculated according to the following formula (1): t :

[0040]

[0041] The conformal P value p of the (ω-1) ion concentration data in the calibration set is calculated according to the following formula (2): j :

[0042]

[0043] Then, the obtained conformal P value p t and conformal P value p j Combined together, a new P value sequence is formed: p t-ω+1 ,p t-ω+2 ,…,p t , when the first monitored process data is observed, the conformal P value sequence includes the P value of the ion concentration data of the first monitored process and the ω-1 data P values ​​in the calibration set; when the second monitored ion concentration data is observed, the conformal P value sequence includes the conformal P values ​​of the ion concentration data of the first and second monitored processes and the ω-2 conformal P values ​​in the calibration set; and so on. The P value sequence contains only ω conformal P values, and changes over time.

[0044] Using the P value in the new P value sequence, the statistic Z of the control chart at each monitoring time t is calculated by the following formula (3): t (ω,λ):

[0045]

[0046] Here are some symbols in the relevant calculation formulas explained: n1 represents the number of historical controlled process ion concentration data in the calibration set, ω represents the time window, and each calculation only uses ω ion concentration samples in the window for calculation, and λ is the smoothing parameter, indicating that over time, the impact of the historical process on the current process gradually decreases.

[0047] S3. Determination of control line:

[0048] The Monte Carlo simulation method is used to approximately obtain the empirical distribution of the statistic, and the α quantile of the empirical distribution is taken as the control line L, resulting in t control lines. Each monitored time t corresponds to a control line, and the maximum value of these t control lines is taken as the final control line L of the control chart.

[0049] Since the distribution of the data is unknown and the reference sample size is insufficient, it is difficult to estimate the distribution of the process data. Therefore, we consider using the Monte Carlo simulation method to approximate the empirical distribution of the statistic and take the α quantile of the distribution as the control line L. In order to facilitate the presentation of this technical solution, the calculation process of the control line is summarized in the following Table 1:

[0050] Table 1: Calculation of control lines

[0051]

[0052] The α quantile should be selected according to the target operating chain length (ARL0) under the controlled state, and the relationship between them is expressed as: ARL0=1 / α, if ARL0=200, then α=0.005; ARL0=370, then α=0.0027; ARL0=500, then α=0.002.

[0053] The calculated control line L is a general result, as long as the parameters n, λ and ω are the same as those used to calculate the control line when calculating the statistics. For example, when calculating the statistic Z during monitoring, t (ω,λ), there are 200 samples in the reference historical controlled data set D, that is, n0=200, and the parameter is selected as λ=0.05, then the control line L=-12.521114 under the same parameter setting can be used.

[0054] S4. Determination of whether the process is under control:

[0055] By comparing the value of the control chart statistic with the final control line L, it is determined whether the ion concentration in the smelting process is within the controlled range. When the control chart statistic is less than the control line L, it means that the ion concentration in the smelting process is not within the controlled range, and the control chart issues an alarm; when the control chart statistic is greater than or equal to the control line L, it means that the process is within the controlled range, the control chart does not issue an alarm, and continues to monitor the subsequent process.

[0056] Since the calculated conformal P will be too small when the process is out of control, the calculated control chart statistic Z t (ω,λ) will also tend to a smaller value. So, when Z t When (ω,λ)<L, the ion concentration in the process is abnormal; when Z t When (ω,λ)≥L, it means that the ion concentration in the process is normal.

[0057] Next, the monitoring process of the entire control chart is summarized in Table 2:

[0058] Table 2: Control chart creation

[0059]

[0060] Next, two examples are used to demonstrate the proposed technical solution.

[0061] Example 1

[0062] Assume that the controlled process ion concentration data obeys the t-distribution with 3 degrees of freedom. First, randomly generate 200 data that obey the t-distribution with 3 degrees of freedom, representing the historical ion concentration data; and 1000 process data to be monitored, representing the ion concentration data to be monitored, and these 1000 data appear in sequence. Divide these 200 data into a training set and a calibration set at a ratio of 1:1, and use the data in the training set to train two score functions S μ0 (X) = (X-μ0) and S knn (X), and use the model to score the data of the calibration set and the process to be monitored, denoted as S(·).

[0063] The corresponding P value and the statistics of the control chart are calculated using equations (1), (2) and (3). The parameters for calculating the statistics are: n0 = 200, n1 = 100, λ = 0.05, and the target controlled average operation chain length is 200, that is, ARL0 = 200. Figure 3 Select the control line L with the same parameters.

[0064] Mean shift δ μ Select: 0, 0.25, 0.5, 1, 2, 3, 4. μ =0 means the process is under control and there is no drift; δ μ ≠0, it means that the process has drifted to varying degrees. μ To study the performance of the control chart, the steps are as follows:

[0065] First, according to the above steps, calculate the statistic Z of the control chart corresponding to each moment based on the data of the process to be monitored. t (ω,λ);

[0066] Secondly, the statistic Z t (ω,λ) is compared with the control line L, which is Figure 3 Select L = -12.5211 and record Z t (ω,λ)<L minimum t, t is the operation chain length of the control chart;

[0067] Finally, repeat the above two steps 1000 times and take the average of the 1000 recorded t, which is the ARL.

[0068] When δ μ=0 is 0, the ARL should be above 200; when δ μ ≠0, that is, δ μ When the other 6 values ​​are selected, the ARL obtained should be as small as possible, indicating that the proposed technical solution for establishing the control chart is more effective. Figure 4 .

[0069] Example 2

[0070] Assume that the controlled process ion concentration data obeys the t-distribution with 3 degrees of freedom. First, randomly generate 200 data that obey the t-distribution with 3 degrees of freedom, representing the historical ion concentration data; and 1000 process data to be monitored, representing the ion concentration data to be detected, and these 1000 data appear in sequence. Divide these 200 data into training set and calibration set in a 1:1 ratio, and use the data in the training set to train a two-score function S μ0 (X) = (X-μ0) and S Me (X) = (X-Me), and use this model to score the data of the calibration set and the process to be monitored, denoted as S μ0 (·) and S M e(·), the control charts obtained under these two scoring functions are denoted as CPE μ0 and CPE Me

[0071] The corresponding P value and the statistics of the control chart are calculated using equations (1), (2) and (3). The parameters for calculating the statistics are: n0 = 200, n1 = 100, λ = 0.05, and the target controlled average operation chain length is 200, that is, ARL0 = 200. Figure 3 Select the control line L with the same parameters.

[0072] Variance drift δ σ Select as: 0, 0.1, 0.2, 0.4, 1.6, 2, 2.5. σ =0 means the process is under control and there is no drift; δ σ ≠0, it means that the process has drifted to varying degrees. σ To study the performance of the control chart, the steps are as follows:

[0073] First, according to the above steps, calculate the statistic Z of the control chart corresponding to each moment based on the data of the process to be monitored. t (ω,λ);

[0074] Secondly, the statistic Z t (ω,λ) is compared with the control line L, which is Figure 3Select L = -12.5211 and record Z t (ω,λ)<L minimum t, t is the operation chain length of the control chart;

[0075] Finally, repeat the above two steps 1000 times and take the average of the 1000 recorded t, which is ARL0.

[0076] When δ σ =0, the ARL should be above 200; when δ σ ≠0, that is, δ σ When the other 6 values ​​are selected, the ARL obtained should be as small as possible, indicating that the proposed technical solution for establishing the control chart is more effective. Figure 5 .

Claims

1. A method for constructing a nonparametric EWMA control chart based on conformal P value, characterized in that: include: S1. Obtaining the data required for the control chart: Get historical controlled process data Y i and the process data to be monitored X t , the distribution of the two sets of data belongs to the ion concentration data of unknown distribution F0 and unknown distribution F1; S2. Determination of statistics of control chart: First calculate the process data X to be monitored t The conformal P value of each data in and the historical controlled process data Y i The conformal P values ​​of the (ω-1) data are then combined to form a new P value sequence. Finally, the new P value sequence is used to calculate the statistics of the control chart at each monitoring time t, where ω is the time window. S3. Determination of control line: Use the Monte Carlo simulation method to approximate the empirical distribution of the statistic, and take the α quantile of the empirical distribution as the control line L, and get t control lines. Each monitoring time t corresponds to a control line, and the maximum value of these t control lines is taken as the final control line L of the control chart; S4. Determination of whether the process is under control: By comparing the value of the control chart statistic with the final control line L, it is determined whether the process is in a controlled state. When the control chart statistic is less than the control line L, it means that the process is out of control and the control chart issues an alarm. When the control chart statistic is greater than or equal to the control line L, it means that the process is in a controlled state, the control chart does not issue an alarm, and continues to monitor the subsequent process.

2. The method for constructing a nonparametric EWMA control chart based on a conformal P value according to claim 1, characterized in that: Historical controlled process data Y i For n0, a historical controlled process data set D is formed, which can be expressed as Divide the historical controlled process data set D into the calibration set D cal and training set D tr , where the calibration set size is n1, the training set size is n0-n1, and the training set D tr Use the trained model to train a scoring function on the data in the calibration set and the process data to be monitored X t Perform an abnormality score, which is recorded as S(·); The statistics of the control chart in S2 are determined, including: Based on the calculated abnormality score S(·), the conformal P value p of the process data to be monitored is calculated according to the following formula (1): t : Based on the calculated abnormality score S(·), the conformal P value p of (ω-1) data in the calibration set is calculated according to the following formula (2): j : The obtained conformal P value p t and p j Combined together, a new P value sequence is formed: p t-ω+1 ,p t-ω+2 ,…,p t , the P value sequence contains only ω P values ​​and changes over time; Using the P value in the new P value sequence, the statistic Z of the control chart at each monitoring time t is calculated by the following formula (3): t (ω,λ):

3. The method for constructing a nonparametric EWMA control chart based on a conformal P value according to claim 2, characterized in that: The score function includes at least one of the following distance functions: The average distance of K nearest neighbors, the average path length, the distance from the hyperplane, the distance from the mean of the training set, the distance from the median of the training set, and the Mahalanobis distance.

4. The method for constructing a nonparametric EWMA control chart based on conformal P value according to claim 2, characterized in that: The α quantile is selected based on the target operating chain length ARL0 under controlled conditions, and the relationship between the two is expressed as: ARL0 = 1 / α.

Images