An industrial data risk assessment method and system based on big data analysis

Abstract

The application discloses an industrial data risk assessment method and system based on big data analysis, relates to the technical field of industrial data risk assessment, and comprises the following steps: collecting industrial data, preprocessing and fusing the industrial data to obtain a fused feature vector; based on the fused feature vector, randomly selecting a feature dimension as a node in an isolation forest algorithm, obtaining a value range, dividing the fused feature vector, obtaining left and right sub-trees, and embedding a performance counter to be initialized; based on the left and right sub-trees, performing a recursive operation to obtain a division path length, calculating an average expected path length and an anomaly score to determine an abnormal point, calculating a prediction error to update the performance counter and perform a tree deletion operation, calculating a Shapley value in combination with the anomaly score, and generating a fused anomaly score; and the application improves the adaptability to industrial environment changes and makes the risk assessment process more comprehensive and accurate.

Description

Technical Field

[0001] This invention relates to the field of industrial data risk assessment technology, and in particular to an industrial data risk assessment method and system based on big data analysis. Background Technology

[0002] With the continuous development of industrial automation and intelligent manufacturing, the amount of data generated by industrial equipment and production environments is growing exponentially. The application of big data technology has made significant progress in the industrial field, especially in equipment monitoring, fault diagnosis, and maintenance optimization. Traditional industrial data analysis methods often rely on human experience and customized rules. When faced with large-scale and complex and ever-changing data, they often fail to achieve efficient analysis and accurate risk assessment. In order to improve the accuracy and efficiency of industrial data processing, more and more companies are beginning to use advanced data analysis methods, such as machine learning, deep learning, and statistical analysis, to conduct risk prediction and fault diagnosis of industrial data. With the maturity of artificial intelligence and big data technologies, data-driven risk assessment methods are gradually becoming an important tool in the industrial field, especially showing great application potential in intelligent manufacturing and predictive maintenance of equipment.

[0003] Existing industrial data risk assessment methods based on big data analytics lack real-time updating and adaptive capabilities, struggle to handle highly complex and dynamically changing data, and are susceptible to noise and outliers when processing high-dimensional data, resulting in insufficient accuracy of assessment results. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an industrial data risk assessment method and system based on big data analysis, which solves the problems of existing industrial data risk assessment methods based on big data analysis lacking real-time updating and adaptive capabilities and being easily affected by noise and outliers, resulting in insufficient accuracy of assessment results.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides an industrial data risk assessment method based on big data analysis, comprising,

[0008] Industrial data is collected, preprocessed, and fused to obtain a fused feature vector.

[0009] The industrial data includes records of industrial equipment operation and maintenance.

[0010] Based on the fused feature vector, a feature dimension is randomly selected as a node in the isolated forest algorithm, and the value range is obtained to divide the fused feature vector to obtain left and right subtrees. The performance counters are then embedded and initialized. Based on the left and right subtrees, a recursive operation is performed to obtain the partition path length. After calculating the average expected path length and the anomaly score to determine the anomaly, the prediction error is calculated to update the performance counters and perform tree deletion operations. The Shapley value is calculated and combined with the anomaly score to generate a fused anomaly score.

[0011] The system concatenates the fused anomaly scores and fused feature vectors, and uses a multi-head attention mechanism to generate contextual risk perception vectors for mapping in the label space to obtain a predicted score. A state transition matrix is ​​randomly generated for initialization, and the objective function of a conditional random field is constructed in combination with the predicted score for iterative optimization to obtain a path score. The Viterbi dynamic programming algorithm is applied to decode the path score iteratively, and the optimal decoded label path is output as the basis for industrial data risk assessment.

[0012] The tag space refers to tag categories, including high risk, medium risk, and low risk;

[0013] Data is stored in a database and displayed using visualization tools.

[0014] As a preferred embodiment of the industrial data risk assessment method based on big data analysis described in this invention, the following steps are taken: the calculation of prediction error updates the performance counter and performs tree deletion operations; the calculation of Shapley value combined with anomaly score generates a fused anomaly score, which is based on the fused feature vector, randomly selects a feature dimension from the feature dimensions as a node in the isolated forest algorithm, uses maximum and minimum operations to obtain the maximum and minimum values ​​of the node as a value range, randomly samples a value from the value range as a partitioning threshold, and partitions all fused feature vectors to obtain a left subtree and a right subtree, and initializes the embedded performance counter for each tree;

[0015] The division refers to the process where, if the value of the same node in the fused feature vector is less than the division threshold, it is assigned to the left subtree; otherwise, it is assigned to the right subtree.

[0016] Recursively operate on the left and right subtrees to obtain the left and right child nodes. Traverse the left and right subtrees and reach the left and right child nodes directly to obtain the partition path length of each fused feature vector.

[0017] Based on all fused feature vectors, the average expected path length is calculated using the natural logarithm function.

[0018] Based on the average expected path length, a normalization factor is calculated. Based on the normalization factor and the partition path length, the anomaly score of the fused feature vector in each tree is calculated. The trees are sorted based on the anomaly scores, and the median is used as the judgment value to determine the anomaly.

[0019] The median refers to the middle number when the median is odd, and the average of the two middle numbers when the median is even.

[0020] Based on the identified outliers, the prediction error is calculated. Based on the prediction error, the performance value of the performance counter is updated. An empirical rule is used to set a replacement threshold. The updated performance value is compared with the replacement threshold. If the updated performance value is less than the replacement threshold, the tree to which the performance counter belongs is deleted. After retraining a new tree, the mean of the updated performance values ​​of all performance counters is calculated using the mean formula as the initial value of the performance counter. Otherwise, it is not deleted.

[0021] Traverse all undeleted trees, extract the corresponding fused feature vectors, and then randomly sample to construct a feature subset. Perform feature masking on the fused feature vectors within this subset to generate a mask vector. Set all features except those in this subset to zero, generating two sets of input vectors. Based on these two sets of input vectors, recalculate the anomaly scores to obtain two sets of anomaly scores S(E). k ,x) and S(E k Using the Shapley value formula, the marginal contribution between the undeleted tree and the fused feature vector is calculated. Then, a Monte Carlo sampling approximation is performed to obtain the Shapley value. Based on the Shapley value and the anomaly score, the fused anomaly score is calculated using a weighted summation formula.

[0022] As a preferred embodiment of the industrial data risk assessment method based on big data analysis described in this invention, the optimal decoding tag path is used as the basis for industrial data risk assessment, which refers to using feature splicing technology to splice the fused anomaly score and the fused feature vector to obtain the enhanced feature vector.

[0023] Based on the enhanced feature vector, a multi-head attention mechanism is used to construct a query, key, and value matrix. After performing a scaling dot product attention operation, the outputs of each attention head are concatenated and then linearly transformed to output a context risk perception vector. The trained linear mapping layer is used to project the context risk perception vector onto the label space to obtain the predicted scores, which are then combined to generate a predicted score vector.

[0024] The tag space refers to tag categories, including high risk, medium risk, and low risk;

[0025] Based on the predicted score vector, a random number generator is used to generate uniformly distributed random values, which are then concatenated to form a state transition matrix for initialization, resulting in a trainable state transition matrix. Based on the predicted score vector and the trainable state transition matrix, the objective function of the conditional random field is defined. The backpropagation algorithm combined with the Adam optimizer is used to update and iterate the parameter combination until the objective function value converges, and the path score is output.

[0026] The parameter combination includes a weight matrix, a bias term, and a state transition matrix;

[0027] The Viterbi dynamic programming algorithm is used in conjunction with the path score sequence to obtain the optimal label path. A loss function is defined and iterated until the loss converges. The optimal decoded label path is then output as the basis for industrial data risk assessment.

[0028] As a preferred embodiment of the industrial data risk assessment method based on big data analysis described in this invention, the preprocessing of collected industrial data refers to obtaining industrial equipment operation and maintenance record data from industrial management equipment through an API interface, cleaning, denoising, and normalizing all data, then using sliding window technology to divide all data, using principal component analysis technology to reduce the dimensionality of the data in each window, and applying feature splicing technology to splice the data in each window to obtain the feature vector of each data in the window.

[0029] As a preferred embodiment of the industrial data risk assessment method based on big data analysis described in this invention, the fusion refers to setting fusion weights based on feature vectors using historical regression analysis, calculating the fusion feature vector within the current window using a weighted summation formula, and recording the feature dimensions.

[0030] As a preferred embodiment of the industrial data risk assessment method based on big data analysis described in this invention, the method involves storing the acquired industrial equipment operation and maintenance record data and the optimal decoded tag sequence in a database, and then adding a unique ID to each type of data in the database and classifying and storing them according to the ID.

[0031] As a preferred embodiment of the industrial data risk assessment method based on big data analysis described in this invention, the optimal decoded label sequence is visualized using a visualization tool, with high risk represented in red, medium risk in black, and low risk in yellow.

[0032] Secondly, this invention provides an industrial data risk assessment system based on big data analysis, comprising:

[0033] The data acquisition and generation module is used to collect industrial data for preprocessing and fusion. It randomly selects feature dimensions as nodes, obtains the value range for partitioning, embeds performance counters for initialization, performs recursive operations to obtain the partition path length, calculates the average expected path length and anomaly score to determine anomalies, calculates the prediction error to update the performance counters and performs tree deletion operations, calculates the Shapley value and combines it with the anomaly score to generate a fused anomaly score.

[0034] The splicing and decoding module is used to perform splicing, generate context risk awareness vectors for label space mapping, randomly generate state transition matrices for initialization, construct objective functions for iterative optimization and decoding iteration, and output the optimal decoded label path;

[0035] The storage and display module is used for storage and display.

[0036] Thirdly, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the industrial data risk assessment method based on big data analysis as described in the first aspect of the present invention.

[0037] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the industrial data risk assessment method based on big data analysis as described in the first aspect of the present invention.

[0038] The beneficial effects of this invention are as follows: By introducing the isolated forest algorithm and combining it with performance counters, elimination mechanisms, and Shapley values, the evaluation strategy can be adjusted in real time according to changes in actual data, thereby improving the adaptability of this invention to changes in the industrial environment. Secondly, the introduction of multi-head attention mechanism, conditional random field algorithm, and Viterbi dynamic programming algorithm makes this invention more comprehensive and accurate in the process of industrial data risk assessment. Attached Figure Description

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a flowchart of the industrial data risk assessment method based on big data analysis in Example 1.

[0041] Figure 2This is a structural diagram of the industrial data risk assessment system based on big data analysis in Example 1.

[0042] Figure 3 This is a flowchart of the anomaly point determination process in Example 1. Detailed Implementation

[0043] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0044] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0045] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0046] Example 1, referring to Figures 1-3 This is the first embodiment of the present invention, which provides an industrial data risk assessment method based on big data analysis, including the following steps:

[0047] S1. Collect industrial data, preprocess and fuse it to obtain a fused feature vector;

[0048] Specifically, collecting and preprocessing industrial data involves obtaining industrial equipment operation and maintenance records from industrial management equipment via API interfaces, cleaning, denoising, and normalizing all data, then using sliding window technology to divide all data, using principal component analysis to reduce the dimensionality of the data in each window, and applying feature splicing technology to splice the data in each window to obtain the feature vector of each data in the window.

[0049] By acquiring real-time operational data and maintenance records from industrial management equipment through API interfaces, the data acquisition becomes efficient and accurate. The application of sliding window technology enables the present invention to perform time series analysis on industrial data. The application of PCA in the present invention helps to solve the curse of dimensionality problem in high-dimensional data analysis. The application of feature stitching technology enables various features within each data window to be comprehensively processed and uniformly represented, thereby generating richer feature vectors.

[0050] Furthermore, fusion refers to setting fusion weights based on feature vectors using historical regression analysis, and calculating the fusion feature vector within the current window using a weighted summation formula, and recording the feature dimensions.

[0051] By setting the weights of fusion features through historical regression analysis, we can combine the trends and patterns of historical data to dynamically adjust the weights of features. The application of the weighted summation formula can reasonably combine the contributions of each feature through weights to generate a fusion feature vector. This can ensure the sensitivity of key features based on the relative importance of each feature.

[0052] S2. Based on the fused feature vector, randomly select feature dimensions as nodes in the isolated forest algorithm, and obtain the value range to divide the fused feature vector to obtain left and right subtrees and embed performance counters for initialization. Based on the left and right subtrees, perform recursive operations to obtain the partition path length, calculate the average expected path length and anomaly score to determine anomalies, calculate the prediction error to update the performance counters and perform tree deletion operations, calculate the Shapley value and combine it with the anomaly score to generate a fused anomaly score.

[0053] Specifically, the prediction error is calculated to update the performance counters and perform tree deletion operations. The Shapley value is calculated and combined with the anomaly score to generate a fused anomaly score. This involves randomly selecting a feature dimension from the feature dimensions as a node in the isolated forest algorithm based on the fused feature vector. The maximum and minimum values ​​of the node are obtained by using the maximum and minimum operations to form a value range. A value is randomly sampled from the value range as a splitting threshold, and all fused feature vectors are split to obtain the left and right subtrees. The performance counter is embedded in each tree for initialization.

[0054] The division refers to the process where, if the value of the same node in the fused feature vector is less than the division threshold, it is assigned to the left subtree; otherwise, it is assigned to the right subtree.

[0055] Recursively operate on the left and right subtrees to obtain the left and right child nodes. Traverse the left and right subtrees and reach the left and right child nodes directly to obtain the partition path length of each fused feature vector.

[0056] Based on all fused feature vectors, the average expected path length is calculated using the natural logarithm function, as follows:

[0057] H(n-1)≈Lin(n-1)+γ,n≥2

[0058] In the formula, H(n-1) represents the average expected path length when the total number of fused feature vectors is n-1, In(·) represents the natural logarithm function (i.e., the natural logarithm function with the natural constant e = 2.718 as the base), and γ represents the Euler-Marcheroni constant (approximately equal to 0.5772).

[0059] Based on the average expected path length, the normalization factor is calculated using the following formula:

[0060]

[0061] In the formula, c represents the normalization factor, and n represents the total number of fused feature vectors;

[0062] Based on the normalization factor and the partition path length, the anomaly score of the fused feature vector in each tree is calculated using the following formula:

[0063]

[0064] In the formula, S r (x) represents the anomaly score of the fused feature vector x in the r-th tree, h r (x) represents the partition path length of the fused feature vector x in the r-th tree;

[0065] The outlier scores are sorted, and the median is used as the criterion for outlier detection. The formula is as follows:

[0066]

[0067] In the formula, y' x θ represents the predicted label of the fused feature vector x, θ represents the decision value, and oud represents otherwise;

[0068] The median refers to the middle number when the median is odd, and the average of the two middle numbers when the median is even.

[0069] Based on the identified outliers, the prediction error is calculated using the following formula:

[0070] E(x) = |y' x -y x |

[0071] In the formula, E(x) represents the prediction error of the fused feature vector x, and y x The baseline label for the fused feature vector x is defined (based on relevant domain knowledge), and |·| represents the absolute value.

[0072] The performance value of the performance counter is updated based on the prediction error, using the following formula:

[0073]

[0074] In the formula, This represents the performance value of the performance counter for the r-th tree after the update, ω represents the exponential decay factor (which can be optimized through grid search), and PC r The initial performance value of the performance counter for the r-th tree (obtained during performance counter initialization);

[0075] The elimination threshold is set using rules of thumb. The updated performance value is compared with the elimination threshold. If the updated performance value is less than the elimination threshold, the tree to which the performance counter belongs is deleted. After retraining the new tree, the mean of the updated performance values ​​of all performance counters is calculated using the mean formula as the initial value of the performance counter. Otherwise, it is not deleted.

[0076] When retraining a new tree, use the fused feature vector within the current time window as training data, and ensure that the window size is consistent with the initial training window.

[0077] Traditional isolated forests are fixed once built and cannot adapt to changes in data distribution. However, this invention tracks the prediction accuracy of each tree in real time through performance counters, eliminates subtrees that have become ineffective due to changes in data distribution, and can adaptively eliminate inferior trees. Furthermore, after eliminating old trees, new trees are retrained, which enables the forest structure to be dynamically updated, maintains sensitivity to the current data pattern, and avoids the computational overhead of maintaining invalid subtrees, thus improving overall efficiency.

[0078] By employing a real-time update and elimination mechanism for performance counters, this invention ensures strong adaptability when processing complex industrial data. The performance counter of each tree records its effectiveness in anomaly detection. A tree is retained only when it meets a specific performance threshold; otherwise, it is eliminated. This elimination mechanism helps avoid overfitting and improves stability and accuracy in practical applications. Furthermore, by calculating the partition path length of each feature vector and combining it with the average expected path length, the degree of anomaly of data points can be effectively quantified. Shorter partition path lengths typically correspond to anomalies, while longer paths indicate that the data points are more normal. This process, through automated calculation and optimization, enables rapid and accurate anomaly detection, greatly improving the real-time performance of industrial data risk assessment. The introduction of a normalization factor solves the problem of differences between different feature dimensions and data distributions, making the anomaly scores of each tree comparable. By standardizing these scores, anomalies can be identified more accurately, improving the accuracy of detection.

[0079] Traverse all trees that have not been deleted, extract the corresponding fused feature vectors, perform random sampling, construct a feature subset, perform feature masking on the fused feature vectors in the feature subset, generate a mask vector, set the features other than the feature subset to zero, and generate two sets of input vectors.

[0080] The two sets of input vectors are one set added to the r-th tree and one set not added to the r-th tree. When setting the vectors to zero, the feature dimension must remain unchanged.

[0081] Based on the two sets of input vectors, the anomaly scores are recalculated to obtain two sets of anomaly scores S(E). k,x) and S(E k ∪{r},x);

[0082] Recalculating the anomaly score involves randomly selecting a feature dimension from the respective feature dimensions of the two sets of input vectors as a node in the Isolation Forest algorithm, obtaining the value range to partition the fused feature vector, obtaining the left subtree and the right subtree, and performing a recursive operation to obtain the left child node and the right child node. The partition path length, the average expected path length, and the normalization factor are obtained, and then the anomaly score of each of the two sets of input vectors is calculated.

[0083] The marginal contribution between the undeleted tree and the fused feature vector is calculated using the Shapley value formula. Then, a Monte Carlo sampling approximation is applied to obtain the Shapley value, as shown in the formula:

[0084]

[0085] In the formula, φ r (x) represents the Shapley value of the fused feature vector x in the r-th tree, K represents the total number of samples, and E k Let S(E) represent the random tree subset at the k-th sampling time, and ∪{r} represent adding the r-th tree to the subset. k ∪{r},x) represents the anomaly score between the r-th tree and the fused feature vector x obtained when the r-th tree is added to the random tree subset at the k-th sampling time, S(E k (x) represents the anomaly score between the random tree subset at the k-th sampling time and the fused feature vector x;

[0086] Based on the Shapley value and outlier score, a weighted summation formula is used to calculate the fused outlier score. The formula is as follows:

[0087]

[0088] In the formula, S ' (x) represents the fusion anomaly score of the fusion feature vector x, and n represents the total number of fusion feature vectors.

[0089] Traditional isolated forests suffer from "black box" characteristics. For example, they construct multiple isolated trees based on randomly partitioning the feature space, and ultimately judge abnormal samples by combining the results of multiple trees. This random partitioning method makes its decision-making process complex and difficult to understand directly, unlike some rule-based or simple distance metric methods that can intuitively provide the basis for judgment. This makes it impossible to explain why a certain sample is judged to be abnormal. In addition, all trees vote equally, ignoring the differences in the quality of subtrees. This invention introduces the Shapley value to quantify the marginal contribution of each tree to the current sample and uses the Shapley value as a weight, so that high-performance trees have more say. By combining the elimination mechanism with the Shapley value, the isolated forest of this invention can achieve a closed loop of "detection-interpretation-optimization".

[0090] Traversing all remaining trees and extracting the corresponding fused feature vectors helps maintain the stability of this invention. Feature masking, by setting specific features to zero, effectively eliminates interference from redundant features and irrelevant information. In industrial data processing, different features contribute differently to anomaly detection and risk assessment. By masking a subset of features and setting irrelevant features to zero, noise from invalid information can be reduced, improving the efficiency of feature selection and the stability of the algorithm. This operation effectively avoids the computational complexity problem caused by excessively high data dimensionality. Shapley value calculation provides a precise measure of the marginal contribution between each tree and the fused feature vector. Through this calculation method, this invention can quantitatively evaluate the specific role of each feature in anomaly detection, and Shapley value can help identify the most critical features, thereby optimizing the data processing process, reducing unnecessary calculations, and improving overall efficiency, especially for industrial equipment. Monitoring and risk assessment can accurately identify key factors affecting performance, helping to discover potential faults and risks earlier. Secondly, the introduction of the Monte Carlo method greatly improves the efficiency of Shapley value calculation. Through sampling approximation, it reduces the time and resources required for calculation, especially suitable for high-dimensional data and large-scale datasets. This method ensures that Shapley values ​​can be estimated efficiently and accurately in complex data environments, providing a reliable basis for subsequent anomaly scoring and risk assessment. By using a weighted summation formula based on the combination of Shapley values ​​and anomaly scores, a fused anomaly score can be calculated, further improving the accuracy of anomaly detection. The contribution of each feature is precisely quantified, avoiding the risk of over-reliance on a single data source or feature, making the final risk assessment results more accurate and reliable. In addition, this can be dynamically updated based on real-time data, providing a more accurate basis for real-time monitoring and fault prediction of industrial equipment.

[0091] S3. Based on the fusion anomaly score and fusion feature vector, the vector is concatenated and a multi-head attention mechanism is used to generate a context risk perception vector for mapping the label space to obtain the predicted score. The state transition matrix is ​​randomly generated for initialization. The objective function of the conditional random field is constructed in combination with the predicted score for iterative optimization to obtain the path score. The Viterbi dynamic programming algorithm is applied to decode the path score iteratively and output the optimal decoded label path as the basis for industrial data risk assessment.

[0092] Specifically, based on the concatenation of fused anomaly scores and fused feature vectors, a multi-head attention mechanism is used to generate a contextual risk perception vector for mapping in the label space to obtain a predicted score. A state transition matrix is ​​randomly generated for initialization, and a conditional random field objective function is constructed in combination with the predicted score for iterative optimization to obtain a path score. The Viterbi dynamic programming algorithm is applied to decode the path score iteratively, and the optimal decoded label path is output as the basis for industrial data risk assessment. Feature concatenation technology is used to concatenate the fused anomaly scores and fused feature vectors to obtain an enhanced feature vector.

[0093] Based on the enhanced feature vector, a multi-head attention mechanism is used to construct a query, key, and value matrix. After performing a scaling dot product attention operation, the outputs of each attention head are concatenated and then linearly transformed to output a context risk perception vector.

[0094] The trained linear mapping layer projects the contextual risk perception vector onto the label space, and the predicted scores are combined to generate a predicted score vector, as shown in the formula:

[0095] P (t) =W s ·h (t) +b s

[0096] In the formula, P (t) W represents the predicted score at time step t. s The weight matrix h represents the label space s (obtained during training of the linear mapping layer). (t) b represents the contextual risk perception vector at time step t. s The bias term representing the label space s (obtained during training of the linear mapping layer);

[0097] The tag space refers to tag categories, including high risk, medium risk, and low risk;

[0098] Based on the predicted score vector, a random number generator is used to generate uniformly distributed random values, which are then concatenated to form a state transition matrix for initialization, thus obtaining a trainable state transition matrix.

[0099] Based on the predicted score vector and the trainable state transition matrix, the objective function of the conditional random field is defined as follows:

[0100]

[0101] In the formula, B(X,T) represents the objective function value given the contextual risk perception vector X and the label sequence Y, and T represents the total number of time steps. Indicates the label category g at time step t. t Predicted score Indicates from label category g t to g t+1 The transition score at that time (obtained from the trainable transition matrix);

[0102] The backpropagation algorithm combined with the Adam optimizer is used to update and iterate the parameter combination until the objective function value converges, and the path score is output.

[0103] The parameter combination includes a weight matrix, a bias term, and a state transition matrix;

[0104] The optimal label path is obtained by combining the Viterbi dynamic programming algorithm with the path score sequence. The loss function is defined as follows:

[0105] L CRF =-(B(X,Y)-logΣ Y'∈H(X) exp(B(X,Y')))

[0106] In the formula, L CRF Let X represent the loss function value, log represent the natural logarithm function, Y' represent all possible label sequences (which can be obtained by combining label categories in the label space using the Cartesian product combination method), H(X) represent the set of all possible label sequences under the context risk perception vector X, ∈ represent belonging to, exp(·) represents the indicator function, log∑exp(·) represents the log-sum-exp operation, and B(X,Y') represent the path score given the context risk perception vector X and the possible label sequence Y'.

[0107] Continue iterating until the loss converges, and then output the optimal decoding tag path as the basis for industrial data risk assessment.

[0108] The introduction of a multi-head attention mechanism enables this invention to learn complex relationships in data from multiple perspectives. Especially when dealing with high-dimensional and complex industrial data, it effectively enhances expressive power. By concatenating and linearly transforming the outputs of multiple attention heads, it can capture deeper patterns at both global and local levels. This allows the invention to more accurately identify and understand the correlations between different features in industrial data risk assessment, improving the accuracy of risk assessment and enhancing fault tolerance and robustness. Through feature concatenation technology, it combines fused anomaly scores and fused feature vectors to generate a more comprehensive enhanced feature vector. This enhanced feature vector has stronger expressive power and can better capture risk signals in the data, thereby improving the ability to detect anomalies in industrial data. This allows the invention to promptly identify potential risk points in complex industrial environments, thus improving the response speed of risk assessment and... By constructing and optimizing the objective function using conditional random fields, risk assessment can more accurately capture the correlation between different risk labels. By optimizing the objective function and dynamically updating the state transition matrix, the prediction of label sequences can be continuously optimized, improving the adaptability to risk changes in industrial data. This operation can handle complex data sequences with time dependencies, greatly improving the stability of risk assessment. Secondly, the introduction of the Viterbi algorithm provides an efficient solution for path score decoding, which can quickly find the optimal decoding path among a large number of possible label sequences. Through this algorithm, the implicit temporal dependencies in complex industrial data can be effectively captured, further improving the accuracy of industrial data risk assessment. Moreover, in this invention, by accurately dividing the label space into three categories—high risk, medium risk, and low risk—accurate risk assessment standards can be provided for each data point in the industrial data.

[0109] S4. Store the data in a database and display it using visualization tools;

[0110] Specifically, the acquired industrial equipment operation and maintenance records and optimal decoded tag sequences are stored in the database. After adding a unique ID to each type of data in the database, the data is classified and stored according to the ID.

[0111] By centrally storing all equipment operation data and maintenance records in the database, unified data management is achieved. By assigning a unique ID to each piece of data and storing it in categories, data for specific devices can be retrieved quickly. Through categorized storage and data ID identification, historical operation and maintenance records for specific devices can be easily extracted from the database.

[0112] Furthermore, visualization tools are used to display the optimal decoded label sequence, with high-risk sequences represented in red, medium-risk sequences in black, and low-risk sequences in yellow.

[0113] By using visualization tools to display the decoded label sequences and marking data of different risk levels with different colors, the efficiency of risk information communication is greatly improved. Through color coding, users can quickly identify high-risk areas and take emergency measures without having to analyze complex numerical data. In addition, the visual impact of colors helps users make quick decisions in emergency situations, effectively avoiding delays caused by traditional data analysis methods.

[0114] This embodiment also provides an industrial data risk assessment system based on big data analysis, including:

[0115] The data acquisition and generation module is used to collect industrial data for preprocessing and fusion. It randomly selects feature dimensions as nodes, obtains the value range for partitioning, embeds performance counters for initialization, performs recursive operations to obtain the partition path length, calculates the average expected path length and anomaly score to determine anomalies, calculates the prediction error to update the performance counters and performs tree deletion operations, calculates the Shapley value and combines it with the anomaly score to generate a fused anomaly score.

[0116] The splicing and decoding module is used to perform splicing, generate context risk awareness vectors for label space mapping, randomly generate state transition matrices for initialization, construct objective functions for iterative optimization and decoding iteration, and output the optimal decoded label path;

[0117] The storage and display module is used for storage and display.

[0118] This embodiment also provides a computer device applicable to the industrial data risk assessment method based on big data analysis, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the industrial data risk assessment method based on big data analysis as proposed in the above embodiment.

[0119] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0120] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the industrial data risk assessment method based on big data analysis as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0121] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for industrial data risk assessment based on big data analysis, characterized in that: include, Industrial data is collected, preprocessed, and fused to obtain a fused feature vector. The industrial data includes records of industrial equipment operation and maintenance. Based on the fused feature vector, a feature dimension is randomly selected as a node in the isolated forest algorithm, and the value range is obtained to divide the fused feature vector to obtain left and right subtrees. The performance counters are embedded and initialized. Based on the left and right subtrees, a recursive operation is performed to obtain the partition path length. After calculating the average expected path length and the anomaly score to determine the anomaly point, the prediction error is calculated to update the performance counters and perform tree deletion operations. The Shapley value is calculated and combined with the anomaly score to generate a fused anomaly score. Specifically, based on the fused feature vector, a feature dimension is randomly selected from the feature dimensions as a node in the isolated forest algorithm. The maximum and minimum values ​​of the node are obtained by using the maximum and minimum operations to form a value range. A value is randomly sampled from the value range as a partitioning threshold, and all fused feature vectors are partitioned to obtain the left subtree and the right subtree. A performance counter is embedded in each tree for initialization. The division refers to the process where, if the value of the same node in the fused feature vector is less than the division threshold, it is assigned to the left subtree; otherwise, it is assigned to the right subtree. Recursively operate on the left and right subtrees to obtain the left and right child nodes. Traverse the left and right subtrees and reach the left and right child nodes directly to obtain the partition path length of each fused feature vector. Based on all fused feature vectors, the average expected path length is calculated using the natural logarithm function. Based on the average expected path length, a normalization factor is calculated. Based on the normalization factor and the partition path length, the anomaly score of the fused feature vector in each tree is calculated. The trees are sorted based on the anomaly scores, and the median is used as the judgment value to determine the anomaly. The median refers to the middle number when the number of outliers is odd, and the average of the two middle numbers when the number of outliers is even. Based on the identified outliers, the prediction error is calculated. Based on the prediction error, the performance value of the performance counter is updated. An empirical rule is used to set a replacement threshold. The updated performance value is compared with the replacement threshold. If the updated performance value is less than the replacement threshold, the tree to which the performance counter belongs is deleted. After retraining a new tree, the mean of the updated performance values ​​of all performance counters is calculated using the mean formula and used as the initial value of the performance counter. Otherwise, it is not deleted. Traverse all undeleted trees, extract the corresponding fused feature vectors, and then randomly sample to construct a feature subset. Perform feature masking on the fused feature vectors within this subset to generate a mask vector. Set all features except those in this subset to zero, generating two sets of input vectors. Based on these two sets of input vectors, recalculate the anomaly scores to obtain two sets of anomaly scores. and , Indicates the first Random tree subsets at each sampling time Indicates the first Add the tree to the subset. Indicates the first The tree joined the first The fused feature vector obtained from the random tree subset during the second sampling Abnormal scores between, Indicates the first Random tree subsets and fused feature vectors at the next sampling time The anomaly score between the trees is calculated using the Shapley value formula to determine the marginal contribution between the undeleted trees and the fused feature vector. Then, a Monte Carlo sampling approximation is used to obtain the Shapley value. Based on the Shapley value and the anomaly score, a weighted summation formula is used to calculate the fused anomaly score. The system concatenates the fused anomaly scores and fused feature vectors, and uses a multi-head attention mechanism to generate contextual risk perception vectors for label space mapping to obtain predicted scores. A randomly generated state transition matrix is ​​used for initialization. Combined with the predicted scores, a conditional random field objective function is constructed for iterative optimization to obtain path scores. The Viterbi dynamic programming algorithm is applied to decode and iterate the path scores, and the optimal decoded label path is output as the basis for industrial data risk assessment. The tag space refers to tag categories, including high risk, medium risk, and low risk; Data is stored in a database and displayed using visualization tools.

2. The industrial data risk assessment method based on big data analysis as described in claim 1, characterized in that: The optimal decoding tag path output as the basis for industrial data risk assessment refers to the use of feature splicing technology to splice the fused anomaly score with the fused feature vector to obtain the enhanced feature vector; Based on the enhanced feature vector, a multi-head attention mechanism is used to construct a query, key, and value matrix. After performing a scaling dot product attention operation, the outputs of each attention head are concatenated and then linearly transformed to output a context risk perception vector. The trained linear mapping layer is used to project the context risk perception vector onto the label space to obtain the predicted scores, which are then combined to generate a predicted score vector. The tag space refers to tag categories, including high risk, medium risk, and low risk; Based on the predicted score vector, a random number generator is used to generate uniformly distributed random values, which are then concatenated to form a state transition matrix for initialization, resulting in a trainable state transition matrix. Based on the predicted score vector and the trainable state transition matrix, the objective function of the conditional random field is defined. The backpropagation algorithm combined with the Adam optimizer is used to update and iterate the parameter combination until the objective function value converges, and the path score is output. The parameter combination includes a weight matrix, a bias term, and a state transition matrix; The Viterbi dynamic programming algorithm is used in conjunction with the path score sequence to obtain the optimal label path. A loss function is defined and iterated until the loss converges. The optimal decoded label path is then output as the basis for industrial data risk assessment.

3. The industrial data risk assessment method based on big data analysis as described in claim 2, characterized in that: The preprocessing of the collected industrial data refers to obtaining the operation and maintenance records of industrial equipment from the industrial management equipment through the API interface, cleaning, denoising, and normalizing all the data, then using the sliding window technique to divide all the data, using principal component analysis to reduce the dimensionality of the data in each window, and applying feature splicing technique to splice the data in each window to obtain the feature vector of each data in the window.

4. The industrial data risk assessment method based on big data analysis as described in claim 3, characterized in that: The fusion refers to setting fusion weights based on feature vectors using historical regression analysis, calculating the fusion feature vector within the current window using a weighted summation formula, and recording the feature dimensions.

5. The industrial data risk assessment method based on big data analysis as described in claim 4, characterized in that: The process of storing data through a database refers to storing the acquired industrial equipment operation and maintenance records and the optimal decoded tag sequence into a database, and then classifying and storing each type of data according to the ID after adding a unique ID to the database.

6. The industrial data risk assessment method based on big data analysis as described in claim 5, characterized in that: The use of visualization tools for display refers to using visualization tools to visualize the optimal decoded tag sequence, with high risk represented in red, medium risk in black, and low risk in yellow.

7. An industrial data risk assessment system based on big data analytics, based on the industrial data risk assessment method based on big data analytics as described in any one of claims 1 to 6, characterized in that: include, The data acquisition and generation module is used to collect industrial data for preprocessing and fusion. It randomly selects feature dimensions as nodes, obtains the value range for partitioning, embeds performance counters for initialization, performs recursive operations to obtain the partition path length, calculates the average expected path length and anomaly score for anomaly point identification, calculates the prediction error, updates the performance counters and performs tree deletion operations, calculates the Shapley value, and combines the anomaly score to generate a fused anomaly score. The splicing and decoding module is used to perform splicing, generate context risk awareness vectors for label space mapping, randomly generate state transition matrices for initialization, construct objective functions for iterative optimization and decoding iteration, and output the optimal decoded label path; The storage and display module is used for storage and display.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the industrial data risk assessment method based on big data analysis as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the industrial data risk assessment method based on big data analysis as described in any one of claims 1 to 6.

Images