A quality abnormality control chart construction method for an industrial big data environment

Abstract

The application discloses a quality abnormality control chart construction method in an industrial big data environment, comprising the following steps: obtaining a training data set, and constructing an isolated binary tree and an isolated tree set by using the training data set; calculating the average isolated depth of sample points in the isolated tree set and designing a sample point abnormality score index; taking the sample point abnormality score index as a monitoring quantity to design control chart parameters, namely constructing a control chart, and applying the constructed control chart to implement monitoring; and the application has the characteristics of unsupervised, high speed, high efficiency and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of manufacturing quality management, and particularly relates to a quality abnormality control chart construction method in an industrial big data environment. BACKGROUND

[0002] A quality control chart is a tool for monitoring process stability and quality changes. It is commonly used in manufacturing and quality management fields to track key parameters of products or processes. By sampling data of product or process quality characteristics, corresponding monitoring statistics are constructed, and the statistical characteristics of the values of the statistics obtained by the sequence sampling are displayed on the quality control chart. When the statistical value exceeds the given statistical limit, a warning signal is issued. When applying control charts to implement quality monitoring, it is usually assumed that the characteristic statistics follow a normal distribution. This technique has played an important role in the quality monitoring of products or processes in the era of low digitalization.

[0003] With the development of information technology and its popularization and application in manufacturing, the digitalization level of product manufacturing processes is getting higher and higher, and a large amount of detection data is generated around the manufacturing process of the product. These data describe the quality characteristics or process state of the product from multiple dimensions, but these high-dimensional big data follow a non-normal distribution or unknown distribution, and the traditional control chart technique is difficult to apply to quality monitoring in such a big data environment. The abnormality detection technology of high-dimensional data is the key to solving the problem.

[0004] Based on machine learning and data mining, experience is learned from industrial big data, and a trained experience model is used to monitor and predict the process, such as online intelligent monitoring of product quality of a steel enterprise production line, quality prediction and control of products delivered in batches by a supplier, identification of quality abnormality patterns, and quality monitoring of time-varying processes. The above methods all belong to supervised learning, which needs to use a large amount of process data related to the monitoring object and label data to complete the training of the algorithm. For the anomaly in the anomaly detection problem, its statistical definition is an outlier, that is, a data sample far away from other observation values. In actual manufacturing processes, the proportion of abnormal data to normal data is seriously unbalanced. Commonly used supervised learning algorithms of machine learning, such as SVM and logistic regression, need a large number of positive and negative samples in the training process of the algorithm, which leads to poor performance of these algorithms for anomaly detection. In the face of continuous real-time high-dimensional data, the complexity and accuracy of the anomaly detection algorithm are very high. With the help of big data analysis technology, constructing a control chart to monitor the anomalies contained in the massive data is a new idea for quality monitoring in an industrial big data environment. SUMMARY

[0005] In order to solve the problems in the prior art, the purpose of the present application is to provide a quality abnormality control chart construction method for an industrial big data environment, which has the characteristics of unsupervised, high speed and high efficiency.

[0006] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application is as follows: a quality abnormality control chart construction method for an industrial big data environment, comprising the following steps:

[0007] Step 1, obtaining a training data set, and constructing an isolated binary tree and an isolated tree set by using the training data set;

[0008] Step 2, calculating the average isolation depth of sample points in the isolated tree set and designing a sample point abnormality score index;

[0009] Step 3, designing control chart parameters by taking the sample point abnormality score index as a monitoring quantity, that is, constructing a control chart, and applying the constructed control chart to implement monitoring.

[0010] As a further improvement of the present application, the step 1 specifically comprises the following steps:

[0011] Step 1.1, obtaining a training data set: obtaining a high-dimensional data set X={x1,x2,…,xs} from a production process, xi={x1,i,x2,i,…,xd,i}, (i=1,2,…,s) is the ith sample point; s},x i ={x i1 ,x i2 ,…,x id};

[0012] Step 1.2, constructing an isolated binary tree t: randomly extracting a sample subset X' with a capacity of N from the training data set, randomly selecting an attribute dimension q (q=1,2,…,d]) and randomly selecting a partition point p in the value range of the selected attribute:

[0013] The sample subset X' is divided into a left subset L and a right subset R by the attribute value, and then L and R are recursively divided until any one of the following conditions is met: the node has only one sample point, or all sample points in the node have the same value on each attribute, thereby obtaining a binary tree t;

[0014] Step 1.3, repeating step 1.2 M times to obtain an isolated tree set T containing M trees, T={t1,t2,…,tM}. M

[0015] As a further improvement of the present application, in step 1.2, for the binary tree t, V is the set of all nodes in t, any node v∈V, the degree deg(v) of the node v refers to the number of child nodes of the node, deg(v) satisfies: ​

[0016] (1) deg(v) e {0, 2}, that is, the degree of each node in the binary tree can only be 0 or 2;

[0017] (2) If deg(v) = 0, the node v is a leaf node;

[0018] (3) If deg(v) = 2, the node v is an internal node;

[0019] (4) For a binary tree with n nodes, the number of internal nodes is n I , and the number of leaf nodes is n L , since each internal node is divided into two, and satisfies the relationship: n I +1 = n L ; therefore, the sum of the degrees of all nodes in the binary tree satisfies:

[0020] As a further improvement of the present application, in step 2, the average isolation depth of the sample points in the isolation tree set is calculated as follows:

[0021] The number of internal nodes experienced from the root node to the external node or leaf node in the process of isolating the data point x i by the isolation tree t j is the isolation depth L(x ij ), and the average value of the isolation depths of the sample points on all isolation trees in the isolation tree set T is the average isolation depth AvgL(x i ):

[0022] As a further improvement of the present application, in step 2, the sample point anomaly score AS is as follows:

[0023]

[0024] Wherein: is the average value of the average isolation depths of all sample points in the data set used to construct the isolation tree set.

[0025] As a further improvement of the present application, in step 3, the control chart parameters include: control chart center line CL, upper control line UCL and lower control line LCL.

[0026] As a further improvement of the present application, the control chart center line upper control line Wherein: {AvgL(x i )} α is the alpha quantile of the data set composed of the average isolation depth values of all sample points in X.

[0027] As a further improvement to the present invention, the monitoring is implemented using the constructed control chart as follows:

[0028] Calculate the anomaly scores of newly collected data sample points, plot the points on the constructed control chart, monitor the abnormal state, and issue an anomaly alarm when the upper control line is exceeded.

[0029] This invention addresses the construction of quality anomaly control charts for multivariate processes. It uses high-dimensional process data as the object, employs an isolation algorithm to construct an isolation tree set, and utilizes this isolation tree set to statistically analyze each sample point x in the training dataset X. i The isolated average path is used to calculate the sample point x. i The abnormal score index AS(x) i The average anomaly score AS(X) of all sample points in the training dataset is used as the center line (CL) of the control chart, and the α quantile of the selected anomaly score index in the training dataset X is used as the control line (UCL) on the control chart. The anomaly scores calculated from the new sample points are used as large point data to implement automatic monitoring and early warning of anomalies.

[0030] The present invention provides a control chart construction scheme based on the average length of isolated paths for quality anomaly monitoring in industrial big data environments. This scheme can effectively solve the problem of detection failure in local high-density areas when existing anomaly detection schemes based on density and distance are applied to the above situations. It features unsupervised operation, high speed, and high efficiency. It implements anomaly monitoring in industrial big data environments in an efficient and intuitive manner in the form of control charts, thus expanding the application scope of quality control charts.

[0031] The beneficial effects of this invention are:

[0032] This invention utilizes big data collected from industrial processes as input, constructs an isolation tree set using an isolation algorithm, and isolates each sample point in the dataset using the isolation tree set to obtain the average isolation path length for each sample point. Based on this, an anomaly score index for sample points is designed. Through exponential transformation, a control chart center line with a constant value of 0.5 and a control chart upper control line with a constant value of 1 are obtained. This enables automatic monitoring of quality big data anomalies in an unsupervised state using control charts, overcoming the limitations of traditional Shewhart control charts that require data to follow a normal distribution, as well as the false alarm problem that exists when multiple charts are used to monitor multivariate processes. At the same time, the detection method based on the average isolation path length of this invention solves the problem of false alarms in locally dense areas when detecting big data anomalies using density-based and distance-based methods. Attached Figure Description

[0033] Figure 1 This is a schematic diagram illustrating the data set segmentation process and the binary tree generation process in an embodiment of the present invention;

[0034] Figure 2 A schematic diagram of the calculation process of the sample point isolation depth in the embodiment of the application;

[0035] Figure 3 A quality big data anomaly detection control chart in the embodiment of the application. DETAILED DESCRIPTION

[0036] The embodiments of the application will be described in detail below with reference to the accompanying drawings.

[0037] Embodiment 1

[0038] A quality anomaly control chart construction method for an industrial big data environment, comprising the following steps:

[0039] Step 1: Obtain a training data set:

[0040] Obtain a high-dimensional data set X = {x1, x2, …, x s}, x i = {x i1 , x i2 , …, x id}, (i = 1, 2, …, s) from a production process, where s is the number of sample points.

[0041] Step 2: Construct an isolation binary tree t:

[0042] Randomly extract a sample subset X' with a capacity of N from the training data set, randomly select an attribute dimension q (q = 1, 2, …, d]), and randomly select a split point p in the value range of the selected attribute:

[0043]

[0044] The sample subset X' is divided into a left subset L and a right subset R by the attribute value, and then L and R are recursively divided until any of the following conditions is met: (i) the node has only one sample point, or (ii) all sample points in the node have the same value on each attribute. The above operation obtains a binary tree t.

[0045] For the binary tree t, V is the set of all nodes in T, and for any node v ∈ V, the degree deg(v) of the node v refers to the number of child nodes of the node, and deg(v) satisfies:

[0046] (1) deg(v) ∈ {0, 2}, i.e., the degree of each node in the binary tree can only be 0 or 2.

[0047] (2) If deg(v) = 0, then the node v is a leaf node.

[0048] (3) If deg(v) = 2, then node v is an internal node.

[0049] (4) For a binary tree with n nodes, the number of internal nodes is n I , and the number of leaf nodes is n L . Since each internal node is divided into two, and satisfies the relationship:

[0050] n I +1 = n L

[0051] Therefore, the sum of the degrees of all nodes in the binary tree satisfies:

[0052]

[0053] Step 3: Constructing the Isolation Tree Set:

[0054] Repeat Step 2 M times to obtain an isolation tree set T containing M trees, T = {t1, t2, …, t M}.

[0055] Step 4: Sample Point Average Isolation Depth Calculation:

[0056] For each isolation tree t j (j = 1, 2, …, M) in the constructed isolation tree set T, the degree of each node v in the tree deg(v) ∈ {0, 2}, i.e. exactly zero or two child nodes. Assuming that each sample point in the data set is different, each instance is isolated to an external node by t j , the number of external nodes is consistent with the capacity of the sample subset X' when constructing the isolation tree, both are N, the number of internal nodes is N-1, and the total number of tree nodes is 2N-1, so the memory requirement is bounded and controllable.

[0057] The process of isolating data point x i to an external node by isolation tree t j , the number of internal nodes experienced from the root node to the external node (or leaf node) is the isolation depth L(x ij ), and the average isolation depth of sample points in all isolation trees in the isolation tree set T is the average isolation depth AvgL(x i ):

[0058]

[0059] Step 5: Sample Point Anomaly Score Index Design:

[0060] Sample point anomaly score index AS (Anormal score) is defined as:

[0061]

[0062] Wherein: is the average value of the average isolation depth of all sample points in the data set used to build the isolation tree set.

[0063] Step 6: Control chart parameter design, i.e. control chart construction:

[0064] In order to apply the control chart to monitor the process collected samples in order to give early warning of abnormalities, the parameters of the control chart need to be designed, including the control center line CL (Central Line) and the upper and lower control lines UCL (Upper control limit) / LCL (Lower control limit). The basic principle of detecting abnormal points by isolating tree partitioning data set is that abnormal points are more easily isolated than normal points, i.e. abnormal points have smaller average isolation depth. Therefore, the above sample point abnormal score index is used as a monitoring quantity, the control chart parameters are designed and the control chart is constructed.

[0065] Control chart center line CL:

[0066]

[0067] When AS(x i ) approaches 1, it means that the average isolation depth of the data point when isolated by the isolation tree set is small, indicating that the data point x i is an abnormal point; when AS(x i ) approaches 0.5, it means that the average isolation depth of the data point when isolated by the isolation tree set is close to the average isolation depth of the entire initial sample set, indicating that the data point x is a normal point.

[0068] In order to accurately identify abnormal samples, the data set {AvgL(x i )(i = 1, 2, …, s)} consisting of the average isolation depth values of all sample points in X is taken, and its α quantile {AvgL(x i ) α , α is related to the quality control level of the process, such as the conventional Shewhart control chart under the control level of ±3σ, α = 0.0273, obtaining the upper control line of the control chart:

[0069]

[0070] There is no upper limit requirement for the average value of the average isolation depth, so the corresponding control chart has no lower control line, i.e. the control chart is a one-sided control chart.

[0071] Step 7: Application of control chart for monitoring:

[0072] Using the method given in step 5, the anomaly score of the newly collected data sample point is calculated, and the point is plotted on the control chart constructed in step 6, and the abnormal state is monitored, and an abnormal alarm is issued when the upper control line is exceeded.

[0073] Embodiment 2

[0074] A quality abnormality control chart construction method for an industrial big data environment, comprising:

[0075] (1) Data set segmentation process: the data set segmentation process and the binary tree generation process are as shown in Figure 1 .

[0076] (2) Sample point isolation depth calculation process:

[0077] Taking a four-dimensional sample point x1(3, 5, 9, 20) as an example, Figure 2 is the process of separating x1 in the isolation tree t1 in the isolation tree set T. The child sample capacity when constructing the isolation tree is consistent with the number of leaf nodes in the tree, which is 4. x1 passes through the root node, the internal node 1, and finally separates to the leaf node. Every time an internal point is passed through, the sample point isolation depth value is increased by 1, and the final external point is guided. The number of internal points passed through in the process of isolating the sample point x1 by t1 is 2, so the isolation depth value L(x 11 ) of the sample point x1 on t1 is 2. Repeat the above process on all M isolation trees in the isolation tree set T to obtain L(x 12 ), … L(x 1M ), and take the average value to obtain the average isolation depth AvgL(x1) of the sample point x1.

[0078] (3) Construction of one-sided control chart:

[0079] There is no upper limit requirement for the average value of the average isolation depth, so the corresponding control chart has no lower control line, that is, the control chart is a one-sided control chart, as shown in Figure 3 . In Phase I, a large amount of data set of the process needs to be collected, which is easy to meet in the industrial big data environment, then the isolation tree set is trained, the anomaly score of each sample point in the training data set is calculated, and the control center line and the control upper limit of the control chart are calculated to establish the control chart; in Phase II, the anomaly score of the newly collected data sample point is calculated first, and the point is plotted on the control chart constructed in Phase I, and the abnormal state is monitored, and an abnormal alarm is issued when the upper control line is triggered.

[0080] The above embodiments only express the specific implementation of the present application, which is described in more detail and in more detail, but it cannot be understood as a limitation on the scope of the patent of the present application. It should be noted that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application.

Claims

1. A method for constructing a quality abnormality control chart in an industrial big data environment, characterized by, The method comprises the following steps: Step 1, obtaining a training data set, and constructing an isolated binary tree and an isolated tree set by using the training data set; The step 1 specifically comprises the following steps: Step 1.1: Obtain the training dataset: Obtain a high-dimensional dataset X = {x1, x2, ..., x} from the production process. s }, x i ={x i1 ,x i2 ,…,x id }, (i = 1, 2, ..., s) represents the i-th sample point; Step 1.2, Constructing Isolation Binary Tree t: Randomly sample a subset X' of size N from the training dataset, randomly select an attribute dimension q (q = 1, 2, …, d]), and randomly select a split point p in the value range of the selected attribute: The sample subset X' is cut into a left subset L and a right subset R by an attribute value, and then L and R are recursively divided until any one of the following conditions is met: the node has only one sample point, or all sample points in the node have the same value on each attribute, thereby obtaining a binary tree t; Step 1.

3. Repeat Step 1.2 M times to get an isolated tree set T containing M trees, T = {t1, t2,..., tM} ; Step 1.

4. For each tree t in T, find the root node r of t, and find the root node r of T. If r is not the root node of T, then replace r with r in t, and replace t with t in T. M} Step 2, calculating the average isolated depth of sample points in the isolated tree set and designing a sample point anomaly score index; In step 2, the average isolated depth of sample points in the isolated tree set is calculated specifically as follows: Data points x i Isolated tree t j The number of internal nodes experienced from the root node to the external node or leaf node in the process of isolating to the external node is the isolation depth L(x ij The average isolation depth AvgL(x i ) of the sample point on all isolated trees in the isolated tree set T is: In step 2, the sample point anomaly score index AS is as follows: wherein: is the average value of the average isolation depth of all sample points in the data set employed for constructing the isolated tree set; Step 3, designing control chart parameters by taking the sample point anomaly score index as a monitoring quantity, that is, constructing a control chart, and implementing monitoring by using the constructed control chart; In step 3, the control chart parameters include a control chart center line CL, an upper control line UCL and a lower control line LCL; Control chart centerline Upper control line wherein: {AvgL(x i )} α is the a-quantile of the dataset of average isolation depth values for all sample points in X.

2. The method for constructing a quality abnormality control chart in an industrial big data environment according to claim 1, characterized by, In step 1.2, for the binary tree t, V is a set of all nodes in t, any node v∈V, the degree deg(v) of the node v refers to the number of child nodes of the node, and deg(v) satisfies: (1) deg(v)∈{0,2}, that is, the degree of each node in the binary tree can only be 0 or 2; (2) if deg(v)=0, the node v is a leaf node; (3) if deg(v)=2, the node v is an internal node; (4) For a binary tree with n nodes, where the number of internal nodes is n I , and the number of leaf nodes is n L , since each internal node is divided into two, and satisfies the relationship: n I +1=n L ; so the sum of the degrees of all nodes in the binary tree satisfies: 3.The method for constructing a quality abnormality control chart in an industrial big data environment according to claim 1, wherein, Implementing monitoring by using the constructed control chart specifically as follows: The anomaly score of the newly collected data sample point is calculated, and the point is drawn on the constructed control chart to monitor the abnormal state, and an abnormal alarm is issued when it exceeds the upper control line.

Images