Quality anomaly control chart construction method used in industrial big data environment

Abstract

The invention discloses a quality anomaly control chart construction method used in an industrial big data environment, and the method comprises the steps: obtaining a training data set, and constructing an isolation binary tree and an isolation tree set through the training data set; calculating the average isolation depth of the sample points in the isolation tree set and designing an abnormal score index of the sample points; designing a control chart parameter by taking the abnormal score index of the sample point as a monitoring quantity, namely constructing a control chart, and implementing monitoring by applying the constructed control chart; the method has the characteristics of no supervision, high speed, high efficiency and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of manufacturing quality management, and in particular to a method for constructing a quality abnormality control chart in an industrial big data environment. Background Art

[0002] The quality control chart is a tool used to monitor process stability and quality changes. It is commonly used in the manufacturing industry and quality management to track key parameters of products or processes. By sampling data on product or process quality characteristics, corresponding monitoring statistics are constructed, and the statistical values ​​and statistical characteristic information obtained from these sequence sampling are displayed on the quality control chart. When the statistical value exceeds the given statistical limit, an early warning signal is issued. When applying control charts to implement quality monitoring, it is usually assumed that the characteristic statistics follow a normal distribution. This technology has played an important role in quality monitoring of products or processes in an era with a low degree of digitization.

[0003] With the development of information technology and its promotion and application in the manufacturing industry, the digital level of product manufacturing process is getting higher and higher, and a large amount of test data is generated around the product manufacturing process. These data describe the quality characteristics or process status of the product from multiple dimensions, but these high-dimensional big data obey non-normal distribution or unknown distribution. Traditional control chart technology is difficult to apply to quality monitoring in this big data environment. Anomaly detection technology for high-dimensional data is the key to solving the problem.

[0004] Based on machine learning and data mining methods, experience is learned from industrial big data, and the trained experience model is used to monitor and predict the process, such as online intelligent monitoring of product quality of steel enterprise production lines, quality prediction and control of batch delivery products from suppliers, identification of quality abnormality patterns, quality monitoring of time-varying processes, etc. The above methods are all supervised learning, which requires a large amount of process data and label data related to the monitored object to complete the training of the algorithm. As for the anomaly in the anomaly detection problem, its statistical definition is an outlier, that is, a data sample far away from other observed values. In the actual manufacturing process, the ratio of abnormal data to normal data is seriously unbalanced. Commonly used supervised learning algorithms of machine learning, such as SVM and logistic regression, require a large number of positive and negative samples in the algorithm training process, resulting in poor performance of such algorithms for anomaly detection. Faced with the continuous generation of real-time high-dimensional data, high requirements are placed on the complexity and accuracy of anomaly detection algorithms. With the help of big data analysis technology, building control charts to monitor anomalies contained in massive data is a new idea for quality monitoring in the industrial big data environment. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the purpose of the present invention is to provide a method for constructing a quality abnormality control chart in an industrial big data environment, which has the characteristics of unsupervised, high speed and high efficiency.

[0006] To achieve the above object, the technical solution adopted by the present invention is: a method for constructing a quality abnormality control chart in an industrial big data environment, comprising the following steps:

[0007] Step 1, obtain a training data set, and use the training data set to construct an isolation binary tree and an isolation tree set;

[0008] Step 2: Calculate the average isolation depth of the sample points in the isolation tree set and design an abnormal score index for the sample points;

[0009] Step 3: Use the sample point abnormal score index as the monitoring quantity to design the control chart parameters, that is, construct the control chart, and use the constructed control chart to implement monitoring.

[0010] As a further improvement of the present invention, the step 1 specifically comprises the following steps:

[0011] Step 1.1, obtain the training data set: obtain a high-dimensional data set X = {x1, x2, ..., x s}, x i ={x i1 ,x i2 ,…,x id}, (i=1,2,…,s) is the i-th sample point;

[0012] Step 1.2, construct the isolation binary tree t: randomly extract a sample subset X′ with a capacity of N from the training data set, randomly select an attribute dimension q (q = 1, 2, ..., d]), and randomly select a split point p within the value range of the selected attribute:

[0013] The sample subset X′ is divided into a left subset L and a right subset R by the attribute value, and then L and R are recursively divided until any of the following conditions are met: the node has only one sample point, or all sample points of the node have the same value on each attribute, thus obtaining a binary tree t;

[0014] Step 1.3: Repeat step 1.2M times to obtain an isolation tree set T containing M trees, T = {t1, t2, ..., t M}.

[0015] As a further improvement of the present invention, in step 1.2, for a binary tree t, V is the set of all nodes in t, and for any node v∈V, the degree deg(v) of the node v refers to the number of child nodes of the node, and deg(v) satisfies:

[0016] (1) deg(v)∈{0,2}, that is, the degree of each node in the binary tree can only be 0 or 2;

[0017] (2) If deg(v) = 0, then node v is a leaf node;

[0018] (3) If deg(v) = 2, then node v is an internal node;

[0019] (4) For a binary tree with n nodes, the number of internal nodes is n I , the number of leaf nodes is n L , since each internal node will be divided into two parts and satisfy the relationship: n I +1=n L ; So the sum of all node degrees in the binary tree satisfies:

[0020] As a further improvement of the present invention, in step 2, the average isolation depth of the sample points in the isolation tree set is calculated as follows:

[0021] Data point x i Isolated Tree j In the process of isolating to an external node, the number of internal nodes that are traversed from the root node to the external node or leaf node is the isolation depth L(x ij ), the average isolation depth of the sample point on all isolation trees in the isolation tree set T is the average isolation depth AvgL(x i ):

[0022] As a further improvement of the present invention, in step 2, the sample point abnormality score index AS is as follows:

[0023]

[0024] in: The average of the mean isolation depths of all sample points in the dataset used to construct the set of isolation trees.

[0025] As a further improvement of the present invention, in step 3, the control chart parameters include: a control chart center line CL, an upper control line UCL and a lower control line LCL.

[0026] As a further improvement of the present invention, the center line of the control diagram Upper control line Where: {AvgL(x i )} α The α quantile of the data set consisting of the average isolation depth values ​​of all sample points in X.

[0027] As a further improvement of the present invention, the control diagram constructed by application is implemented as follows:

[0028] Calculate the anomaly score of the newly collected data sample points, plot the points on the constructed control chart, monitor the abnormal state, and issue an abnormal alarm when it exceeds the upper control line.

[0029] The present invention aims at constructing a quality abnormality control chart for a multivariate process, takes the high-dimensional collected data of the process as the object, constructs an isolation tree set using the isolation algorithm, and uses the isolation number set to statistically analyze each sample point x in the training data set X. i The isolated average path is used to calculate the sample point x i The abnormal score index AS(x i ), the average anomaly score AS(X) of all sample points in the training data set is taken as the center line (CL) of the control chart, the α quantile of the sample of the anomaly score indicator in the training data set X is taken as the upper control line (UCL) of the control chart, and the anomaly score calculated from the new sample point is used as the large point data to implement automatic monitoring and early warning of anomalies.

[0030] The control chart construction scheme based on the average length of isolation paths suitable for quality anomaly monitoring in an industrial big data environment of the present invention can effectively solve the detection failure problem in local high-density areas when existing anomaly detection schemes based on density, distance, etc. are applied to the above situation, and has the characteristics of unsupervised, high speed, and high efficiency; it implements anomaly monitoring in an industrial big data environment efficiently and intuitively in the form of a control chart, expanding the application scope of quality control charts.

[0031] The beneficial effects of the present invention are:

[0032] The present invention uses big data collected from industrial processes as input, uses an isolation algorithm to construct an isolation tree set, uses the isolation tree set to isolate each sample point of the data set, obtains the average isolation path length of each sample point, and designs an abnormal score index of the sample point based on this. Through exponential transformation, a control chart center line with a constant value of 0.5 and an upper control line of the control chart with a constant value of 1 are obtained, thereby realizing automatic monitoring of abnormalities of quality big data with the help of control charts in an unsupervised state, breaking through the limitation of traditional Shewhart control charts that data must obey normal distribution, and the problem of underreporting when multiple charts are jointly implemented to monitor multivariate processes. At the same time, the present invention solves the problem of false alarms in local dense areas when detecting abnormalities of big data based on density and distance methods based on the detection method based on the average isolation path length. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 Schematic diagram of the data set segmentation process and binary tree generation process in an embodiment of the present invention;

[0034] Figure 2 Schematic diagram of the calculation process of the isolation depth of a sample point in an embodiment of the present invention;

[0035] Figure 3 This is a quality big data anomaly detection control chart in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0037] Example 1

[0038] A method for constructing a quality abnormality control chart in an industrial big data environment comprises the following steps:

[0039] Step 1: Get the training dataset:

[0040] Get a high-dimensional dataset X from the production process = {x1, x2, ..., x s}, x i ={x i1 ,x i2 ,…,x id}, (i=1,2,…,s) is the i-th sample point;

[0041] Step 2: Construct an isolated binary tree t:

[0042] Randomly extract a sample subset X′ of capacity N from the training data set, randomly select an attribute dimension q (q = 1, 2, …, d]), and randomly select a split point p within the value range of the selected attribute:

[0043]

[0044] The sample subset X′ is divided into a left subset L and a right subset R by the attribute value, and then L and R are recursively divided until any of the following conditions are met: (i) the node has only one sample point, or (ii) all sample points of the node have the same value on each attribute. The above operation results in a binary tree t.

[0045] For a binary tree t, V is the set of all nodes in T. For any node v∈V, the degree of node v deg(v) refers to the number of child nodes of the node, and deg(v) satisfies:

[0046] (1) deg(v)∈{0,2}, that is, the degree of each node in the binary tree can only be 0 or 2.

[0047] (2) If deg(v) = 0, then node v is a leaf node.

[0048] (3) If deg(v) = 2, then node v is an internal node.

[0049] (4) For a binary tree with n nodes, the number of internal nodes is n I , the number of leaf nodes is n L , since each internal node will be divided into two parts and satisfy the relationship:

[0050] n I +1=n L

[0051] So the sum of all node degrees in a binary tree satisfies:

[0052]

[0053] Step 3: Build the isolation tree set:

[0054] Repeat the steps 2M times to obtain an isolation tree set T containing M trees, T = {t1, t2, ..., t M}.

[0055] Step 4: Calculation of average isolation depth of sample points:

[0056] For each isolation tree t in the isolation tree set T that has been built j (j=1,2,…M), the degree of each node v in the tree is deg(v)∈{0,2}, that is, it has exactly zero or two child nodes. Assume that each sample point in all data sets is different and each instance is t j Isolate to an external node. At this time, the number of external nodes is consistent with the capacity of the sample subset X′ when building the isolation tree, which is N. The number of internal nodes is N-1, and the total number of nodes in the tree is 2N-1. Therefore, the memory requirement is bounded and controllable.

[0057] Data point x i Isolated Tree j In the process of isolating to an external node, the number of internal nodes that a node passes through from the root node to the external node (or leaf node) is the isolation depth L (x ij ), the average isolation depth of the sample point on all isolation trees in the isolation tree set T is the average isolation depth AvgL(x i ):

[0058]

[0059] Step 5: Sample point abnormal score indicator design:

[0060] The definition of sample point abnormal score indicator AS (Anormal score) is:

[0061]

[0062] in: The average of the mean isolation depths of all sample points in the dataset used to construct the set of isolation trees.

[0063] Step 6: Control chart parameter design, i.e. control chart construction:

[0064] In order to use control charts to monitor process samples for early warning of abnormalities, it is necessary to design control chart parameters, including the control center line CL (Central Line) and the upper and lower control lines UCL (Upper control limit) / LCL (Lower control limit). The basic principle of detecting abnormal points by segmenting the data set through isolation trees is that abnormal points are easier to be isolated than normal points, that is, abnormal points have a smaller average isolation depth. Therefore, the present invention uses the above-mentioned sample point abnormality score index as the monitoring quantity, designs the control chart parameters and constructs the control chart.

[0065] Control chart center line CL:

[0066]

[0067] When AS(x i ) is close to 1, indicating that the average isolation depth of the data point when isolated by the isolation tree set is small, indicating that the data point x i is an abnormal point; when AS(x i ) is close to 0.5, which means that the average isolation depth of the data point when isolated by the isolation tree set is close to the average isolation depth of the initial sample set as a whole, indicating that the data point x is a normal point.

[0068] In order to accurately identify abnormal samples, the average isolation depth values ​​of all sample points in X constitute the data set {AvgL(x i )(i=1,2,…,s)}, take its α quantile {AvgL(x i )} α , α is related to the quality control level of the process. For example, for a conventional Shewhart control chart at a ±3σ control level, α=0.0273, and the upper control line of the control chart is obtained:

[0069]

[0070] There is no upper limit requirement for the average value of the average isolation depth, so the corresponding control chart has no lower control line, that is, the control chart is a one-sided control chart.

[0071] Step 7: Apply control diagrams to implement monitoring:

[0072] Using the method given in step 5, calculate the abnormal score of the newly collected data sample point, and draw the point on the control chart constructed in step 6 to monitor the abnormal state and issue an abnormal alarm when it exceeds the upper control line.

[0073] Example 2

[0074] A method for constructing a quality abnormality control chart in an industrial big data environment, comprising:

[0075] (1) Data set segmentation process: The data set segmentation process and binary tree generation process are as follows: Figure 1 As shown;

[0076] (2) Calculation process of sample point isolation depth:

[0077] Take a four-dimensional sample point x1(3,5,9,20) as an example, Figure 2 The figure shows the process of isolating x1 through the isolation tree t1 in the isolation tree set T. The subsample capacity when constructing the isolation tree is consistent with the number of leaf nodes in the tree, which is 4. x1 passes through the root node and the inner node 1 in turn, and finally separates to the leaf node. Every time it passes through an inner point, the isolation depth value of the sample point increases by 1 until it reaches the final outer point. The number of inner points that the sample point x1 passes through during the isolation process of t1 is 2, so the isolation depth value L(x1) of the sample point x1 on t1 is 11 )=2. Repeat the above process on all M isolation trees in the isolation tree set T, and obtain L(x 12 ),…L(x 1M ), and taking the average value is the average isolation depth AvgL(x1) of the sample point x1.

[0078] (3) Construction of one-sided control chart:

[0079] There is no upper limit requirement for the average value of the average isolation depth, so the corresponding control chart has no lower control line, that is, the control chart is a one-sided control chart, such as Figure 3 As shown. In Phase I, a large number of process data sets need to be collected, which is easy to meet in the industrial big data environment. Then, the isolation tree set is trained, the abnormal score of each sample point in the training data set is calculated, and the control center line and upper control limit of the control chart are calculated to establish the control chart; in Phase II, the abnormal score of the newly collected data sample point is first calculated, and the point is drawn on the control chart constructed in Phase I to monitor the abnormal state and issue an abnormal alarm when the upper control line is triggered.

[0080] The above-mentioned embodiments only express the specific implementation of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for constructing a quality abnormality control chart in an industrial big data environment, characterized in that: The following steps are involved: Step 1, obtain a training data set, and use the training data set to construct an isolation binary tree and an isolation tree set; Step 2: Calculate the average isolation depth of the sample points in the isolation tree set and design an abnormal score index for the sample points; Step 3: Design control chart parameters using the sample point abnormality score index as the monitoring quantity, that is, construct a control chart, and use the constructed control chart to implement monitoring.

2. The method for constructing a quality abnormality control chart in an industrial big data environment according to claim 1, characterized in that: The step 1 specifically comprises the following steps: Step 1.1, obtain the training data set: obtain a high-dimensional data set X = {x1, x2, ..., x s }, x i ={x i1 ,x i2 ,…,x id }, (i=1,2,…,s) is the i-th sample point; Step 1.2, construct the isolation binary tree t: randomly extract a sample subset X′ with a capacity of N from the training data set, randomly select an attribute dimension q (q = 1, 2, ..., d]), and randomly select a split point p within the value range of the selected attribute: The sample subset X′ is divided into a left subset L and a right subset R by the attribute value, and then L and R are recursively divided until any of the following conditions are met: the node has only one sample point, or all sample points of the node have the same value on each attribute, thus obtaining a binary tree t; Step 1.3: Repeat step 1.2M times to obtain an isolation tree set T containing M trees, T = {t1, t2, ..., t M }.

3. The method for constructing a quality abnormality control chart in an industrial big data environment according to claim 2, characterized in that: In step 1.2, for a binary tree t, V is the set of all nodes in t. For any node v∈V, the degree of node v, deg(v), refers to the number of child nodes of the node. deg(v) satisfies: (1) deg(v)∈{0,2}, that is, the degree of each node in the binary tree can only be 0 or 2; (2) If deg(v) = 0, then node v is a leaf node; (3) If deg(v) = 2, then node v is an internal node; (4) For a binary tree with n nodes, the number of internal nodes is n I , the number of leaf nodes is n L , since each internal node will be divided into two parts and satisfy the relationship: n I +1=n L ; So the sum of all node degrees in the binary tree satisfies:

4. The method for constructing a quality abnormality control chart in an industrial big data environment according to claim 3 is characterized in that: In step 2, the average isolation depth of the sample points in the isolation tree set is calculated as follows: Data point x i Isolated Tree j In the process of isolating to an external node, the number of internal nodes that are traversed from the root node to the external node or leaf node is the isolation depth L(x ij ), the average isolation depth of the sample point on all isolation trees in the isolation tree set T is the average isolation depth AvgL(x i ):

5. The method for constructing a quality abnormality control chart in an industrial big data environment according to claim 4 is characterized in that: In step 2, the sample point abnormality index AS is as follows: in: The average of the mean isolation depths of all sample points in the dataset used to construct the set of isolation trees.

6. The method for constructing a quality abnormality control chart in an industrial big data environment according to claim 5, characterized in that: In step 3, the control chart parameters include: a control chart center line CL, an upper control line UCL, and a lower control line LCL.

7. The method for constructing a quality abnormality control chart in an industrial big data environment according to claim 6, characterized in that: Control Chart Center Line Upper control line Where: {AvgL(x i )} α The α quantile of the data set consisting of the average isolation depth values ​​of all sample points in X.

8. The method for constructing a quality abnormality control chart in an industrial big data environment according to claim 7, characterized in that: The control charts built by the application are implemented as follows: Calculate the anomaly score of the newly collected data sample points, plot the points on the constructed control chart, monitor the abnormal state, and issue an abnormal alarm when it exceeds the upper control line.

Images