Device anomaly detection system and method based on cloud-edge collaboration mode

Abstract

The present invention specifically relates to a device anomaly detection system and method based on a cloud-edge collaborative mode. The system includes: a device layer, which is used to obtain the operating data of the device; a cloud, where several trained fault detection models are deployed; the fault detection model is a BiLSTM‑VAE model; an edge server, which has multiple parallel edge nodes for downloading corresponding fault detection models from the cloud; the edge server performs weighted calculations on the outputs of the fault detection models of each edge node to determine whether the device has a fault; the cloud generates decision information and sends it to the edge server; the edge server sends the decision information to the device layer for the device layer to perform corresponding actions. The present invention also discloses a device anomaly detection method. The present invention can improve the performance of device anomaly detection by integrating and complementing multiple fault detection models with different hidden layer dimensions, and does not require the construction of a model structure specific to the problem scenario, and does not require prior experience and professional knowledge.

Description

Technical Field

[0001] The present invention relates to the technical field of device anomaly detection, and in particular to a device anomaly detection system and method based on a cloud-edge collaboration mode. Background Art

[0002] Cloud-edge collaboration is a system architecture that deploys infrastructure resources in a distributed manner and then manages them in a unified manner. Unlike cloud manufacturing, which concentrates all computing services in cloud data centers, cloud-edge collaboration uses the Internet of Things to achieve real-time perception of distributed industrial equipment. The data information of terminal devices is uploaded to the nearby edge computing nodes through intelligent gateways. The edge nodes process the collected measurement data in the form of time series through parallel local computing services, thereby realizing autonomous control decisions of machinery and equipment and personalized automatic regulation of production processes. It provides smart factories with intelligent services such as processing quality inspection, manufacturing resource scheduling, and logistics control, thereby enhancing the production control capabilities and comprehensive competitiveness of traditional discrete manufacturing companies.

[0003] The cloud-edge collaborative architecture model can not only provide real and effective data for smart factories in real time and accurately, enhance the real-time control response speed of terminal equipment, but also adapt to the real-time analysis of massive data in advanced industrial production processes, and reduce the computing pressure and network load of cloud data centers. However, as the scale of enterprise production continues to expand, the number of terminal mechanical equipment and edge computing devices is also increasing, resulting in the topology of the cloud-edge collaborative network becoming more complex and the incidence of interference factors increasing exponentially, leading to the increasing difficulty of system management and maintenance costs.

[0004] When edge computing devices encounter abnormal conditions, such as network failure, server failure, configuration error, server overload, external attack, etc., it will not only affect the efficiency and stability of data processing, but also restrict the efficient execution of the corresponding terminal equipment tasks of the node, such as visual inspection of product defects, manufacturing resource scheduling, etc. The failure of terminal mechanical equipment has a greater impact on the production quality of products and the safety of workers' operations, resulting in a decrease in the product qualification rate of production lines, an increase in fault repair costs and order delay delivery losses. Therefore, achieving real-time monitoring of the operating status of terminal mechanical equipment and edge computing equipment, and supporting intelligent prediction and alarm of faults in production equipment and edge nodes, is the basis for the efficient operation of smart factories and cloud-edge collaborative networks.

[0005] Establishing machine learning models through historical equipment operation data to achieve real-time status monitoring, anomaly detection, and fault diagnosis of industrial equipment operation processes has become a hot topic for the intelligent improvement of manufacturing enterprises. Existing research results have achieved good application results in detecting equipment anomalies using machine learning. However, the method of calculating the residual of the original time series data and the reconstructed data for anomaly detection, represented by the LSTM-AE model, has the problem that the reconstruction error adaptation interval is narrow, resulting in poor performance of the algorithm in indicators such as accuracy (precision) and false alarm rate (FAR), resulting in poor accuracy of equipment anomaly detection. At the same time, the residual threshold and neural network structure of the existing model rely on manual parameter adjustment, such as the need to build a model structure specific to the problem scenario or require prior experience and professional knowledge, which has poor adaptability and cannot be applied on a large scale in scenarios such as smart factories with a wide variety of equipment types and fault categories, resulting in poor practicality of equipment anomaly detection. Therefore, how to design a method that can improve the practicality and accuracy of equipment anomaly detection is a technical problem that needs to be solved urgently. Summary of the invention

[0006] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is: how to provide a device anomaly detection method based on a cloud-edge collaborative mode, so as to improve the device anomaly detection performance through the integrated complementarity between multiple fault detection models with different hidden layer dimensions, and there is no need to build a model structure specific to the problem scenario, and no prior experience and professional knowledge is required, thereby improving the practicality and accuracy of device anomaly detection.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] The device anomaly detection system based on the cloud-edge collaborative mode includes:

[0009] The device layer is used to obtain the operation data of the device;

[0010] In the cloud, several trained fault detection models are deployed;

[0011] The fault detection model is a BiLSTM-VAE model obtained by embedding a bidirectional long short-term memory network into the encoding layer and decoding layer of a variational autoencoder. The hidden layer dimensions of each fault detection model are different.

[0012] An edge server having a plurality of parallel edge nodes respectively used for downloading corresponding fault detection models from the cloud;

[0013] The edge server uses the operating data as the input of each edge node fault detection model, and performs weighted calculation on the output of each edge node fault detection model to determine whether the device has a fault;

[0014] When a device fails, the cloud analyzes the failure using real-time and historical operation data, generates decision information, and sends it to the edge server;

[0015] The edge server sends the decision information to the device layer so that the device layer can perform corresponding actions.

[0016] Preferably, when there is no fault in the device, the edge server uploads the operation data to the cloud at a preset frequency as historical operation data; when there is a fault in the device, the edge server uploads the operation data containing the fault information to the cloud at a frequency higher than the preset frequency as real-time operation data.

[0017] The present invention also discloses a device anomaly detection method based on a cloud-edge collaborative mode, and the device anomaly detection system based on a cloud-edge collaborative mode of the present invention is implemented, specifically including the following steps:

[0018] S1: The device layer obtains the operating data of the device;

[0019] S2: Each edge node of the edge server downloads the trained fault detection model from the cloud;

[0020] The fault detection model is a BiLSTM-VAE model obtained by embedding a bidirectional long short-term memory network into the encoding layer and decoding layer of a variational autoencoder. The hidden layer dimensions of each fault detection model are different.

[0021] S3: The edge server uses the operating data as the input of each edge node fault detection model, and performs weighted calculation on the output of each edge node fault detection model to determine whether the device has a fault;

[0022] S4: When a device fails, the cloud analyzes the failure based on real-time and historical operation data, generates decision information, and sends it to the edge server;

[0023] S5: The edge server sends the decision information to the device layer so that the device layer can perform corresponding actions.

[0024] Preferably, in step S2, the cloud trains the fault detection model through the following steps:

[0025] S201: Obtain a sample data set and perform data preprocessing, and then divide the preprocessed data into a training set consisting of all normal labeled data, a test set consisting of all abnormal labeled data, and a validation set consisting of a mixture of normal data and abnormal data;

[0026] S202: Setting K0 fault detection models with different hidden layer dimensions, and then using the training set to train each fault detection model to achieve feature learning of the normal state data of the equipment, and generate an initial residual threshold matrix;

[0027] S203: Eliminate K1 fault detection models with the worst performance to obtain K2=K0-K1 fault detection models;

[0028] S204: Initializing the reconstruction error weight matrix and the residual threshold weight matrix corresponding to the remaining fault detection models through a chaotic strategy;

[0029] S205: Using the whale optimization algorithm to solve the optimal reconstruction error weight matrix and residual threshold weight matrix, and then obtain the fault detection model k j and its corresponding reconstruction error weight Residual Threshold Weight and residual threshold σ j ;

[0030] S206: Test the performance of each fault detection model through the test set. If the performance meets the standard, end the training and verify the training effect of each fault detection model through the verification set; otherwise, return to step S204.

[0031] Preferably, in step S201, data preprocessing includes Z-score normalization processing and feature enhancement processing; statistical features and time series prediction features are extracted during feature enhancement processing; time series prediction features include predicted values ​​of exponentially weighted moving average method, and the difference between the predicted value and the original value;

[0032] Among them, the predicted value of the exponentially weighted moving average method is calculated by the following formula:

[0033]

[0034] Where: y i represents the predicted value of the exponentially weighted moving average method at time i; x i Represents the original value at time i; t represents the total step length of the time series data; the coefficient α represents the rate of weighted decrease. The larger its value is, the faster the weight of the early data decreases.

[0035] Preferably, in step S203, each fault detection model is evaluated by using the false alarm rate and the missed alarm rate: the smaller the false alarm rate and the missed alarm rate, the better the effect of the fault detection model; the K1 fault detection models at the end of the comprehensive evaluation are eliminated, and the K2=K0-K1 fault detection models with the highest comprehensive evaluation are selected;

[0036] in,

[0037]

[0038] Where: FAR represents the false alarm rate; MAR represents the missed alarm rate; TP represents the number of times the fault detection model accurately detects abnormal points; FP represents the number of times the fault detection model identifies normal points as abnormal points; FN represents the number of times the fault detection model identifies abnormal points as normal points; TN represents the number of times the fault detection model accurately identifies normal points.

[0039] Preferably, in step S205, for the whale optimization algorithm:

[0040] The original fixed probability value of the whale optimization algorithm is replaced by a dynamically changing AP value;

[0041] AP=min_AP+max_AP*t / t max ;

[0042] Where: t is the current iteration number; t max Maximum number of iterations;

[0043] The iterative formula of the whale's position in the stage of contraction and encirclement of prey is expressed as:

[0044]

[0045]

[0046]

[0047] C = 1.5*sin(2π*rand3)+0.5;

[0048] Where: rand1, rand2 and rand3 are all random numbers between [0,1]; denote the positions of humpback whale i at iterations t and t+1, respectively; is an individual randomly selected from the current population; D represents the individual Iterative moving distance; A represents individual Iterative moving direction; a represents the nonlinear convergence factor; C is a random value in the range [0, 2];

[0049] The iterative formula for the whale position in the hunting phase is expressed as:

[0050]

[0051] l = a2 + rand4;

[0052]

[0053] Where: is a randomly selected individual from the current elite solution set; They represent the position of humpback whale i at the tth and t+1th iterations respectively; rand4 is a random number between [0,1]; b is a constant that limits the shape and size of the spiral; l is improved from a random number between [-1,1] to a coefficient that decays nonlinearly with the algorithm iteration process, which is used to control the shape of the logarithmic spiral; a2 represents the nonlinear convergence factor; β represents the adjustment of the collection speed;

[0054] The search formula for new prey is expressed as:

[0055]

[0056]

[0057]

[0058]

[0059]

[0060] Where: represents the position of humpback whale i at the t+1th iteration; is a randomly selected individual from the current elite solution set; μ and v follow the standard normal distribution: A represents the displacement step; Levy(γ) represents the Levy flight function; γ is the coefficient constant; a3 represents the nonlinear convergence factor; φ represents the step size of the Levy flight; Γ represents the gamma function; exp represents exponential operation.

[0061] Preferably, in step S206, the performance of the fault detection model is evaluated by three evaluation indicators: precision, recall rate and F-score;

[0062] in,

[0063]

[0064]

[0065] Where: Precision represents precision; TP represents the number of times the fault detection model accurately detects abnormal points; FP represents the number of times the fault detection model identifies normal points as abnormal points; Recall represents the recall rate; FN represents the number of times the fault detection model identifies abnormal points as normal points.

[0066] Preferably, in step S3, the edge server performs weighted calculation on the reconstruction errors output by the fault detection models of each edge node to obtain a comprehensive reconstruction error; then performs weighted calculation based on the residual thresholds corresponding to each fault detection model to obtain a comprehensive residual threshold; finally, the comprehensive reconstruction error and the comprehensive residual threshold are compared to determine whether the device has a fault.

[0067] Preferably, the comprehensive reconstruction error is calculated by the following formula:

[0068]

[0069] Where: Loss i represents the time series data X of K fault detection models for running data i The comprehensive reconstruction error of loss j,i represents the jth fault detection model k j For the time series data X of the running data i The reconstruction error of Denotes the fault detection model k j The reconstruction error weight of

[0070] The comprehensive residual threshold is calculated by the following formula:

[0071]

[0072] Where: j Denotes the fault detection model k j The residual threshold of Denotes the fault detection model k j The residual threshold weight of ;

[0073] When Loss l When <δ, the time series data X of the operation data i Normal, the device is normal; Loss l ≥δ, the time series data X of the running data i Abnormal. The device is abnormal.

[0074] The device anomaly detection method based on the cloud-edge collaboration mode in the present invention has the following beneficial effects:

[0075] The present invention uses the operating data as the input of each edge node fault detection model through the edge server, and performs weighted calculation on the output of each edge node fault detection model to determine whether the device has a fault, so that the device anomaly detection performance can be improved through the integrated complementarity between multiple parallel edge nodes arranged on the edge server and fault detection models with different hidden layer dimensions, thereby improving the accuracy of device anomaly detection. In addition, the fault detection model in the present invention (i.e., the BiLSTM-VAE model) can be directly solved by existing algorithms (such as the whale optimization algorithm), without relying on manual parameter adjustment, without the need to build a model structure specific to the problem scenario, and without prior experience and professional knowledge, thereby improving the practicality of device anomaly detection.

[0076] At the same time, the present invention obtains the operation data of the equipment through the equipment layer, and downloads the fault detection model from the cloud through the edge server to determine whether the equipment has a fault, so that real-time equipment operation status analysis and fault detection can be performed through the edge server deployed inside the factory, thereby improving the real-time performance of equipment abnormality detection.

[0077] In addition, the present invention detects equipment failures by downloading fault detection models from the cloud through the edge server, generates decision information through fault analysis based on real-time operation data and historical operation data through the cloud, and sends it to the edge server for the device layer to execute corresponding actions. That is, a working mode is formed in which the edge node is responsible for equipment operation status detection under daily circumstances, and the cloud is responsible for emergency handling under fault conditions. This can not only reduce the amount of data interacting between the edge server and the cloud center, avoid frequent occurrence of problems such as communication network fluctuations, transmission data loss, and user privacy leakage, but also improve the real-time control response speed of production site equipment, avoid the cloud computing center processing computing tasks with excessive data volume and complexity, thereby further improving the work efficiency of the cloud-edge collaborative network. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] In order to make the purpose, technical solution and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0079] Figure 1 This is the network architecture diagram of the device anomaly detection system based on the cloud-edge collaboration model;

[0080] Figure 2 It is a logical block diagram of the device anomaly detection method based on the cloud-edge collaboration mode;

[0081] Figure 3 This is a schematic diagram of the internal structure of LSTM;

[0082] Figure 4 Schematic diagram of the composition structure of Bi-LSTM;

[0083] Figure 5 Schematic diagram of the composition structure of the automatic encoder (VAE);

[0084] Figure 6 This is a schematic diagram of the structure of the fault detection (BiLSTM-VAE) model;

[0085] Figure 7 Flowchart for training fault detection model;

[0086] Figure 8 This is a schematic diagram of the SKAB data collection environment;

[0087] Fig. 9 It is a schematic diagram of the TEP simulation environment;

[0088] Fig.10 This is the BiLSTM-VAE-WOA detection result diagram;

[0089] Fig.11 This is the detection graph of BiLSTM-VAE1;

[0090] Fig.12 This is the BiLSTM-VAE2 detection graph;

[0091] Fig.13 is the reconstruction error density distribution map. DETAILED DESCRIPTION

[0092] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but only represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0093] It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. In the description of the present invention, it should be noted that the orientation or position relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inside", "outside", etc. is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invention product is usually placed when used, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation of the present invention. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance. In addition, the terms "horizontal", "vertical", etc. do not mean that the components are absolutely horizontal or suspended, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted. In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0094] The following is a further detailed description through specific implementation methods:

[0095] Embodiment 1:

[0096] This embodiment discloses a device anomaly detection system based on a cloud-edge collaboration mode.

[0097] like Figure 1 As shown in the figure, the device anomaly detection system based on the cloud-edge collaborative mode includes:

[0098] The device layer is used to obtain the operation data of the device;

[0099] In this embodiment, by deploying data devices such as sensors (industrial cameras, RFID scanners, etc.) for collecting and executing terminal data, networking devices (such as PLC controllers embedded with 5G modules, two-way communication gateways with intelligent data aggregation modules, etc.) and other data devices at the production site, a large number of underlying Internet of Things (terminal) nodes, namely the device layer, are constructed, which are responsible for collecting and transmitting the operating data of industrial production equipment, providing a data basis for equipment fault detection in smart factories.

[0100] In the cloud (central cloud), several trained fault detection models are deployed;

[0101] The fault detection model is a BiLSTM-VAE model obtained by embedding a bidirectional long short-term memory network (Bi-LSTM) into the encoding layer and decoding layer of a variational auto-encoder (VAE). The hidden layer dimensions of each fault detection model are different.

[0102] In this embodiment, since the training and debugging of the deep learning model and the swarm intelligence evolution algorithm (i.e., the fault detection model) require a large amount of computing resources to ensure that the algorithm model achieves a high accuracy within a limited time, the edge server with limited computing power and storage space cannot be responsible for the training tasks with huge computing volume. Therefore, the present invention has a cloud computing center equipped with a large amount of computing resources responsible for model training and updating, thereby ensuring the fault detection accuracy of the fault detection model.

[0103] An edge server having multiple parallel edge (computing) nodes respectively used to download corresponding fault detection models from the cloud;

[0104] In this embodiment, the edge computing node is also responsible for uploading various production data to the cloud for aggregation and storage for use in subsequent quality traceability and other modules.

[0105] In order to reduce the load of data transmission in the communication network, when there is no fault in the equipment, the edge server uploads the operation data to the cloud at a preset frequency as historical operation data; when there is a fault in the equipment, the edge server uploads the operation data containing the fault information to the cloud at a frequency higher than the preset frequency as real-time operation data.

[0106] The edge server uses the operating data as the input of each edge node fault detection model, and performs weighted calculation on the output of each edge node fault detection model to determine whether the device has a fault;

[0107] When a device fails, the cloud analyzes the failure using real-time and historical operation data, generates decision information, and sends it to the edge server;

[0108] The edge server sends the decision information to the device layer so that the device layer can perform corresponding actions.

[0109] In this embodiment, after the edge server sends the decision information to the equipment layer, the relevant workshop managers make corresponding on-site decisions based on the decision information, and then the edge node of the edge server drives the PLC and other controllers to adjust the operating status of the production line through the on-site decision.

[0110] The present invention uses the operating data as the input of each edge node fault detection model through the edge server, and performs weighted calculation on the output of each edge node fault detection model to determine whether the device has a fault, so that the device anomaly detection performance can be improved through the integrated complementarity between multiple parallel edge nodes arranged on the edge server and fault detection models with different hidden layer dimensions, thereby improving the accuracy of device anomaly detection. In addition, the fault detection model in the present invention (i.e., the BiLSTM-VAE model) can be directly solved by existing algorithms (such as the whale optimization algorithm), without relying on manual parameter adjustment, without the need to build a model structure specific to the problem scenario, and without prior experience and professional knowledge, thereby improving the practicality of device anomaly detection.

[0111] At the same time, the present invention obtains the operation data of the equipment through the equipment layer, and downloads the fault detection model from the cloud through the edge server to determine whether the equipment has a fault, so that real-time equipment operation status analysis and fault detection can be performed through the edge server deployed inside the factory, thereby improving the real-time performance of equipment abnormality detection.

[0112] In addition, the present invention detects equipment failures by downloading fault detection models from the cloud through the edge server, generates decision information through fault analysis based on real-time operation data and historical operation data through the cloud, and sends it to the edge server for the device layer to execute corresponding actions. That is, a working mode is formed in which the edge node is responsible for equipment operation status detection under daily circumstances, and the cloud is responsible for emergency handling under fault conditions. This can not only reduce the amount of data interacting between the edge server and the cloud center, avoid frequent occurrence of problems such as communication network fluctuations, transmission data loss, and user privacy leakage, but also improve the real-time control response speed of production site equipment, avoid the cloud computing center processing computing tasks with excessive data volume and complexity, thereby further improving the work efficiency of the cloud-edge collaborative network.

[0113] Specific:

[0114] 1. Whale Optimization Algorithm (WOA Algorithm)

[0115] The whale optimization algorithm has the advantages of simple principle, few parameter settings, and strong optimization performance. However, the whale optimization algorithm only relies on one parameter to adjust the conversion between the early global search and the later local optimization, which makes it difficult to achieve a good balance. It has the problem of fast convergence speed and easy to fall into the local optimum.

[0116] Therefore, the present invention replaces the original fixed probability value (50%) of the whale optimization algorithm with a dynamically changing AP value; sets p to a random number uniformly distributed in the range of [0,1] to make a selection decision between shrinkage predation and random search:

[0117] AP=min_AP+max_AP*t / t max .

[0118] Where: t is the current iteration number; t max Maximum number of iterations;

[0119] (1) Contraction predation (p<AP)

[0120] a) Surround the prey (|A|≥1)

[0121] The whale population will surround the prey based on each other's positions. Table 1 shows the original and improved formulas for the whale position iteration in the stage of shrinking and surrounding the prey.

[0122] Table 1 Comparison of whale position iteration formulas

[0123]

[0124] The iterative formula of the whale's position in the stage of contraction and encirclement of prey is expressed as:

[0125]

[0126]

[0127]

[0128] C = 1.5*sin(2π*rand3)+0.5;

[0129] Where: rand1, rand2 and rand3 are all random numbers between [0,1]; denote the positions of humpback whale i at iterations t and t+1, respectively; is an individual randomly selected from the current population; D represents the individual Iterative moving distance; A represents individual Iterative moving direction; a represents the nonlinear convergence factor, and the value change can autonomously adjust the time point of the algorithm's early and late iteration center of gravity shift; C is a random value in the range [0, 2], which expands the global search range of the population.

[0130] The improved formula of the present invention indicates that individual whales will randomly select other whales as guides for position iteration. The reason for this improvement is that the original formula and spiral predation both perform position iteration based on the current optimal individual of the whale group, which is prone to clustering, resulting in a decrease in the population diversity of the whale position, thereby helping to improve the accuracy of device anomaly detection. At the same time, the nonlinear change degree of the coefficient vectors a and A can be adjusted through the coefficient β to balance the center of gravity of the early and late iterations of the algorithm.

[0131] b) Hunting behavior (|A|<1)

[0132] The position iteration formula in the hunting phase imitates the spiral predation behavior of whale groups in bubble net attacks. The position of the whale individual with the best current fitness is set as the prey position, and the prey is approached and trapped along the spiral path.

[0133] Table 2 Comparison of whale position iteration formulas

[0134]

[0135] The iterative formula for the whale position in the hunting phase is expressed as:

[0136]

[0137] l = a2 + rand4;

[0138]

[0139] Where: is a randomly selected individual from the current elite solution set; They represent the position of humpback whale i at the tth and t+1th iterations respectively; rand4 is a random number between [0,1]; b is a constant that limits the shape and size of the spiral; l is improved from a random number between [-1,1] to a coefficient that decays nonlinearly with the algorithm iteration process, which is used to control the shape of the logarithmic spiral; a2 represents the nonlinear convergence factor, and its value change can adjust when the algorithm ends the early global candidate solution screening and when it enters the later stage for fine search in the local area; β represents the adjustment of the collection speed.

[0140] (2) Random search predation (p ≥ AP)

[0141] In addition to approaching known prey, whales also conduct searches for new prey.

[0142] Table 3 Comparison of search formulas for new prey

[0143]

[0144] The search formula for new prey is expressed as:

[0145]

[0146]

[0147]

[0148]

[0149] Where: represents the position of humpback whale i at the t+1th iteration; is a randomly selected individual from the current elite solution set; μ and v follow the standard normal distribution: A represents the displacement step size; Levy (γ) represents the Levy flight function, which generates a random value to perturb the position of the elite individual; γ is a coefficient constant; a3 represents the nonlinear convergence factor, and the value change can adjust when the algorithm ends the early global candidate solution screening and when it enters the later stage to perform fine search in the local area; φ represents the step size of the Levy flight; Γ represents the gamma function; exp represents exponential operation.

[0150] The original formula of the whale optimization algorithm uses a position iteration method consistent with the stage of shrinking and surrounding prey, which reduces the global search ability of the algorithm. However, the improved formula of the present invention uses the Levy flight algorithm to randomly perturb the position of the whale elite individual, and adjusts the value of the random perturbation by the coefficients A and a3 that perform numerical nonlinear attenuation during the algorithm solution process, which helps the algorithm expand the global search space in the early stage, and at the same time improves the local fine optimization ability of the algorithm in the later stage.

[0151] 2. Bidirectional Long Short-Term Memory Network (BiLSTM)

[0152] Long Short Term Memory Networks (LSTM) is an optimized model of RNN. The LSTM model is mainly composed of a forget gate, an input gate, and an output gate. The input part includes C t-1 、h t-1 and x t , the internal structure of LSTM is as follows Figure 3 shown.

[0153] The role of forget gates is to determine the old state information C t-1 The output is a vector f with the same length as the cell state vector t , the value range is between [0,1]:

[0154] f t =σ*(W f [h t-1 ,x t ]+b f );

[0155] The role of the incoming gate is to determine h t-1 and x t What information is stored in the cell? This is mainly divided into two parts. The Sigmoid layer determines which values ​​will be updated, and the weighted tanh layer determines the candidate vectors added to the cell state.

[0156] i t =σ(Wi [h t-1 ,x t ]+b i );

[0157]

[0158] By f t Expression of C t-1 The expected replacement part, i t Expressed new information The expected additions to update the status information:

[0159]

[0160] The output gate is used to determine the final output information h t , the Sigmoid layer passes through o t Determine the output and normalize the value to [-1, 1] using the tanh function:

[0161] o t =σ(W o [h t-1 ,x t ]+b o );

[0162] h t =o t *tanh(C t );

[0163] Since in the unidirectional LSTM network, the model processes time series data in a positive direction, C t-1 With h t-1 It contains historical information before time t, but does not learn data after time t. This one-way learning method increases the risk of overfitting the model and reduces the accuracy of reconstructed data.

[0164] The bidirectional long short-term memory network (BiLSTM) combines an LSTM network that moves from the beginning to the end of the sequence and an LSTM network that moves from the end of the sequence to the beginning of the sequence. The output at time t comprehensively considers several historical input information and several future input information, such as Figure 4 shown.

[0165] In the forward layer, the input time series X is calculated forward from time 1 to time 2 = {X1, X2…, X i}The hidden state of each time step, get X i The forward output Then the Backward layer performs reverse calculation from time i to time 1 to obtain X iThe reverse output of

[0166] The forward layer and the backward layer are connected to the output layer together, combining the output results of the two LSTM networks at each moment. and The final output is obtained by adding, averaging or concatenating.

[0167] 3. Variational Autoencoder (VAE)

[0168] A typical autoencoder (AE) is a three-layer neural network that uses the encoding layer to transform the original n-dimensional input data Mapped to the m-dimensional intermediate hidden layer, extracting nonlinear features The decoding layer then reconstructs the hidden layer vector into n-dimensional output data The training process is to minimize the input X i and output Y i The reconstruction error is the optimization target of the model parameters, and the calculation formula of the reconstruction error loss is as follows:

[0169]

[0170] Traditional autoencoders are prone to overfitting. The overall structure of the variational autoencoder (VAE) is similar to that of the autoencoder AE. The difference is that the model perturbs the features extracted from the middle hidden layer to generate new samples that are similar but different from the training data. Figure 5 shown.

[0171] The encoding layer outputs two feature vectors m = {m1, m2..., m m} and σ={σ1,σ2...,σ m}, and then generate a vector data e={e1,e2...,e m}, finally execute the formula to get the final hidden layer feature vector C i =exp(σ i )×e i +m i .

[0172] In actual production environments, the time when abnormal situations occur during daily equipment operation is much shorter than the time when it operates normally. The frequency of abnormalities is very low, resulting in insufficient abnormal data available for analysis. In addition, because the equipment working scenarios of smart factories in the cloud-edge collaborative mode are constantly changing dynamically, business types are constantly updated and upgraded, resulting in a diversity of abnormal fault types. As an unsupervised learning algorithm, VAE only needs to be trained based on normal data samples, without learning the characteristics of fault data. By comparing the difference between the original data X and the reconstructed data Y, if the residual between the reconstructed data and the original data is too large (that is, the reconstruction error is too large), it is considered that there may be an abnormality.

[0173] Although VAE can solve the problem of sample size imbalance between normal data and abnormal data, it is difficult to identify the residual interval of normal data and abnormal data, and has poor learning effect on time series data. Therefore, the present invention embeds the Bi-LSTM network with better time series data processing effect into the encoding layer and decoding layer of VAE to construct a BiLSTM-VAE model, whose structure is as follows: Figure 6 shown.

[0174] BiLSTM-VAE model through X i and Y i The residual Loss i The abnormality is determined by comparing it with the reconstruction threshold δ. The calculation formula of the threshold is as follows:

[0175] σ=p·max(||XY|| 2 );

[0176]

[0177] Among them, δ is the reconstruction threshold, α and β represent the 90% quantile of the reconstruction error value set of the BiLSTM-VAE model for normal data and the 10% quantile of the reconstruction error value set of abnormal data, respectively.

[0178] The hyperparameters of the BiLSTM-VAE model are difficult to set. Inappropriate hyperparameters will lead to low model diagnostic accuracy and high false alarm rate. In addition, a single fault detection model has the problem that the reconstruction error intervals of normal data and abnormal data are not clearly differentiated. Therefore, the present invention sets K BiLSTM-VAE fault detection models with different hidden layer dimensions to jointly obtain residuals. Each fault detection model will generate a set of reconstructed data. Finally, the fault detection result of each time step data is the weighted sum of the reconstruction errors of each fault detection model under the weight matrix.

[0179] During the specific implementation process, the edge server performs weighted calculation on the reconstruction errors output by the fault detection models of each edge node to obtain a comprehensive reconstruction error; then performs weighted calculation based on the residual thresholds corresponding to each fault detection model to obtain a comprehensive residual threshold; finally, the comprehensive reconstruction error and the comprehensive residual threshold are compared to determine whether there is a fault in the device.

[0180] The comprehensive reconstruction error is calculated by the following formula:

[0181]

[0182] Where: Loss i represents the time series data X of K fault detection models for running data i The comprehensive reconstruction error of loss j,i represents the jth fault detection model k j For the time series data X of the running data i The reconstruction error of Denotes the fault detection model k j The reconstruction error weight of

[0183] The comprehensive residual threshold is calculated by the following formula:

[0184]

[0185] Where: j Denotes the fault detection model k j The residual threshold of Denotes the fault detection model k j The residual threshold weight of ;

[0186] When Loss l When <δ, the time series data X of the operation data i Normal, the device is normal; Loss l ≥δ, the time series data X of the running data i Abnormal. The device is abnormal.

[0187] The present invention uses the edge server to use the operating data as the input of each edge node fault detection model, and performs weighted calculation on the output of each edge node fault detection model to determine whether there is a fault in the equipment, so that the equipment abnormality detection performance can be improved through the integrated complementarity between multiple parallel edge nodes arranged on the edge server and the fault detection models with different hidden layer dimensions, that is, the fault detection result of each time step data is the weighted sum of the reconstruction errors of each fault detection model under the weight matrix, thereby further improving the accuracy of equipment abnormality detection.

[0188] At the same time, the fault detection model in the present invention (i.e., the BiLSTM-VAE model) can be directly solved by the whale optimization algorithm, without relying on manual parameter adjustment, without the need to build a model structure specific to the problem scenario, and without the need for prior experience and expertise, thereby further improving the practicality of equipment anomaly detection.

[0189] In the specific implementation process, Figure 7 As shown in the figure, the cloud trains the fault detection model through the following steps:

[0190] S201: Obtain a sample data set and perform data preprocessing, and then divide the preprocessed data into a training set consisting of all normal labeled data, a test set consisting of all abnormal labeled data, and a validation set consisting of a mixture of normal data and abnormal data;

[0191] S202: Setting K0 fault detection models with different hidden layer dimensions, and then using the training set to train each fault detection model to achieve feature learning of the normal state data of the equipment, and generate an initial residual threshold matrix;

[0192] S203: Eliminate K1 fault detection models with the worst performance to obtain K2=K0-K1 fault detection models;

[0193] S204: Initializing the reconstruction error weight matrix and the residual threshold weight matrix corresponding to the remaining fault detection models through a chaotic strategy;

[0194] In this embodiment, chaos in the chaos strategy refers to a non-periodic motion form that is unique to nonlinear systems and widely exists. Chaos: There is no universal and strict definition of chaos. It is generally believed that the random motion state that is not caused by random external factors but directly obtained by deterministic equations (internal factors) is called chaos.

[0195] Chaos theory mainly studies the behavior of dynamic systems that are particularly sensitive to initial states. Its main properties include: ergodicity (the ability to traverse all states in a certain area), randomness, and regularity.

[0196] The purpose of the chaos strategy is to generate a chaotic population by using a one-dimensional chaotic map, specifying a random initial value, and iterating continuously to generate a series of continuous points.

[0197] S205: The optimal reconstruction error weight matrix and residual threshold weight matrix are obtained by solving the Whale Optimization Algorithm (WOA), and then the fault detection model k is obtained. j and its corresponding reconstruction error weight Residual Threshold Weight and residual threshold σ j;

[0198] S206: Test the performance of each fault detection model through the test set. If the performance meets the standard, end the training and verify the training effect of each fault detection model through the verification set; otherwise, return to step S204.

[0199] In this embodiment, if the training effect of the fault detection model verified by the verification set is not good, the parameters of the fault detection model are adjusted and modified.

[0200] In the specific implementation process, data preprocessing includes Z-score standardization processing (using one-hot encoding technology to discretely digitize fault labels) and feature enhancement processing; when enhancing features, statistical features (first-order differences) and time series prediction features are extracted. Time series prediction features include the predicted value of the exponentially weighted moving average (EWMA) method and the difference between the predicted value and the original value;

[0201] Among them, the predicted value of the exponentially weighted moving average method is calculated by the following formula:

[0202]

[0203] Where: y i represents the predicted value of the exponentially weighted moving average method at time i; x i Represents the original value at time i; t represents the total step length of the time series data; the coefficient α represents the rate of weighted decrease. The larger its value is, the faster the weight of the early data decreases.

[0204] In the specific implementation process, the false alarm rate and missed alarm rate are used to evaluate each fault detection model: the smaller the false alarm rate and missed alarm rate, the better the effect of the fault detection model; the K1 fault detection models at the end of the comprehensive evaluation are eliminated, and the K2 = K0-K1 fault detection models with the highest comprehensive evaluation are selected;

[0205] in,

[0206]

[0207] Where: FAR represents the false alarm rate; MAR represents the missed alarm rate; TP represents the number of times the fault detection model accurately detects abnormal points; FP represents the number of times the fault detection model identifies normal points as abnormal points; FN represents the number of times the fault detection model identifies abnormal points as normal points; TN represents the number of times the fault detection model accurately identifies normal points.

[0208] In the specific implementation process, the performance of the fault detection model is evaluated by three evaluation indicators: precision, recall, and F-score;

[0209] in,

[0210]

[0211]

[0212] Where: Precision represents precision; TP represents the number of times the fault detection model accurately detects abnormal points; FP represents the number of times the fault detection model identifies normal points as abnormal points; Recall represents the recall rate; FN represents the number of times the fault detection model identifies abnormal points as normal points.

[0213] Since the training and debugging of deep learning models and swarm intelligence evolution algorithms (i.e., fault detection models) require a large amount of computing resources to ensure that the algorithm model achieves a high level of accuracy within a limited time, edge servers with limited computing power and storage space cannot be responsible for training tasks with large amounts of computing. Therefore, the present invention uses a cloud computing center equipped with a large amount of computing resources to train and update the model, thereby ensuring the fault detection accuracy of the fault detection model.

[0214] Embodiment 2:

[0215] This embodiment discloses a device anomaly detection method based on a cloud-edge collaboration mode, which is implemented based on the device anomaly detection system based on a cloud-edge collaboration mode in Embodiment 1.

[0216] like Figure 2 As shown, the device anomaly detection method based on the cloud-edge collaboration mode specifically includes the following steps:

[0217] S1: The device layer obtains the operating data of the device;

[0218] S2: Each edge node of the edge server downloads the trained fault detection model from the cloud;

[0219] The fault detection model is a BiLSTM-VAE model obtained by embedding a bidirectional long short-term memory network (Bi-LSTM) into the encoding layer and decoding layer of a variational auto-encoder (VAE). The hidden layer dimensions of each fault detection model are different.

[0220] S3: The edge server uses the operating data as the input of each edge node fault detection model, and performs weighted calculation on the output of each edge node fault detection model to determine whether the device has a fault;

[0221] S4: When a device fails, the cloud analyzes the failure based on real-time and historical operation data, generates decision information, and sends it to the edge server;

[0222] S5: The edge server sends the decision information to the device layer so that the device layer can perform corresponding actions.

[0223] In this embodiment, after the edge server sends the decision information to the equipment layer, the relevant workshop managers make corresponding on-site decisions based on the decision information, and then the edge node of the edge server drives the PLC and other controllers to adjust the operating status of the production line through the on-site decision.

[0224] The present invention uses the operating data as the input of each edge node fault detection model through the edge server, and performs weighted calculation on the output of each edge node fault detection model to determine whether the device has a fault, so that the device anomaly detection performance can be improved through the integrated complementarity between multiple parallel edge nodes arranged on the edge server and fault detection models with different hidden layer dimensions, thereby improving the accuracy of device anomaly detection. In addition, the fault detection model in the present invention (i.e., the BiLSTM-VAE model) can be directly solved by existing algorithms (such as the whale optimization algorithm), without relying on manual parameter adjustment, without the need to build a model structure specific to the problem scenario, and without prior experience and professional knowledge, thereby improving the practicality of device anomaly detection.

[0225] At the same time, the present invention obtains the operation data of the equipment through the equipment layer, and downloads the fault detection model from the cloud through the edge server to determine whether the equipment has a fault, so that real-time equipment operation status analysis and fault detection can be performed through the edge server deployed inside the factory, thereby improving the real-time performance of equipment abnormality detection.

[0226] In addition, the present invention detects equipment failures by downloading fault detection models from the cloud through the edge server, generates decision information through fault analysis based on real-time operation data and historical operation data through the cloud, and sends it to the edge server for the device layer to execute corresponding actions. That is, a working mode is formed in which the edge node is responsible for equipment operation status detection under daily circumstances, and the cloud is responsible for emergency handling under fault conditions. This can not only reduce the amount of data interacting between the edge server and the cloud center, avoid frequent occurrence of problems such as communication network fluctuations, transmission data loss, and user privacy leakage, but also improve the real-time control response speed of production site equipment, avoid the cloud computing center processing computing tasks with excessive data volume and complexity, thereby further improving the work efficiency of the cloud-edge collaborative network.

[0227] Finally, since the training and debugging of deep learning models and swarm intelligence evolution algorithms (i.e., fault detection models) require a large amount of computing resources to ensure that the algorithm model achieves a high level of accuracy within a limited time, edge servers with limited computing power and storage space cannot be responsible for training tasks with large amounts of computing. Therefore, the present invention uses a cloud computing center equipped with a large amount of computing resources to train and update the model, thereby ensuring the fault detection accuracy of the fault detection model.

[0228] Specific:

[0229] 1. Whale Optimization Algorithm (WOA Algorithm)

[0230] The whale optimization algorithm has the advantages of simple principle, few parameter settings, and strong optimization performance. However, the whale optimization algorithm only relies on one parameter to adjust the conversion between the early global search and the later local optimization, which makes it difficult to achieve a good balance. It has the problem of fast convergence speed and easy to fall into the local optimum.

[0231] Therefore, the present invention replaces the original fixed probability value (50%) of the whale optimization algorithm with a dynamically changing AP value; sets p to a random number uniformly distributed in the range of [0,1] to make a selection decision between shrinkage predation and random search:

[0232] AP=min_AP+max_AP*t / t max ;

[0233] Where: t is the current iteration number; t max Maximum number of iterations.

[0234] (1) Contraction predation (p<AP)

[0235] a) Surround the prey (|A|≥1)

[0236] The whale population will surround the prey based on each other's positions. Table 1 shows the original and improved formulas for the whale position iteration in the stage of shrinking and surrounding the prey.

[0237] Table 1 Comparison of whale position iteration formulas

[0238]

[0239] The iterative formula of the whale's position in the stage of contraction and encirclement of prey is expressed as:

[0240]

[0241]

[0242]

[0243] C = 1.5*sin(2π*rand3)+0.5;

[0244] Where: rand1, rand2 and rand3 are all random numbers between [0,1]; denote the positions of humpback whale i at iterations t and t+1, respectively; is an individual randomly selected from the current population; D represents the individual Iterative moving distance; A represents individual Iterative moving direction; a represents the nonlinear convergence factor, and the value change can autonomously adjust the time point of the algorithm's early and late iteration center of gravity shift; C is a random value in the range [0, 2], which expands the global search range of the population.

[0245] The improved formula of the present invention indicates that individual whales will randomly select other whales as guides for position iteration. The reason for this improvement is that the original formula and spiral predation both perform position iteration based on the current optimal individual of the whale group, which is prone to clustering, resulting in a decrease in the population diversity of the whale position, thereby helping to improve the accuracy of device anomaly detection. At the same time, the nonlinear change degree of the coefficient vectors a and A can be adjusted through the coefficient β to balance the center of gravity of the early and late iterations of the algorithm.

[0246] b) Hunting behavior (|A|<1)

[0247] The position iteration formula in the hunting phase imitates the spiral predation behavior of whale groups in bubble net attacks. The position of the whale individual with the best current fitness is set as the prey position, and the prey is approached and trapped along the spiral path.

[0248] Table 2 Comparison of whale position iteration formulas

[0249]

[0250] The iterative formula for the whale position in the hunting phase is expressed as:

[0251]

[0252] l = a2 + rand4;

[0253]

[0254] Where: is a randomly selected individual from the current elite solution set; They represent the position of humpback whale i at the tth and t+1th iterations respectively; rand4 is a random number between [0,1]; b is a constant that limits the shape and size of the spiral; l is improved from a random number between [-1,1] to a coefficient that decays nonlinearly with the algorithm iteration process, which is used to control the shape of the logarithmic spiral; a2 represents the nonlinear convergence factor, and its value change can adjust when the algorithm ends the early global candidate solution screening and when it enters the later stage for fine search in the local area; β represents the adjustment of the collection speed.

[0255] (2) Random search predation (p ≥ AP)

[0256] In addition to approaching known prey, whales also conduct searches for new prey.

[0257] Table 3 Comparison of search formulas for new prey

[0258]

[0259] The search formula for new prey is expressed as:

[0260]

[0261]

[0262]

[0263]

[0264] Where: represents the position of humpback whale i at the t+1th iteration; is a randomly selected individual from the current elite solution set; μ and v follow the standard normal distribution: A represents the displacement step size; Levy (γ) represents the Levy flight function, which generates a random value to perturb the position of the elite individual; γ is a coefficient constant; a3 represents the nonlinear convergence factor, and the value change can adjust when the algorithm ends the early global candidate solution screening and when it enters the later stage to perform fine search in the local area; φ represents the step size of the Levy flight; Γ represents the gamma function; exp represents exponential operation.

[0265] The original formula of the whale optimization algorithm uses a position iteration method consistent with the stage of shrinking and surrounding prey, which reduces the global search ability of the algorithm. However, the improved formula of the present invention uses the Levy flight algorithm to randomly perturb the position of the whale elite individual, and adjusts the value of the random perturbation by the coefficients A and a3 that perform numerical nonlinear attenuation during the algorithm solution process, which helps the algorithm expand the global search space in the early stage, and at the same time improves the local fine optimization ability of the algorithm in the later stage.

[0266] 2. Bidirectional Long Short-Term Memory Network (BiLSTM)

[0267] Long Short Term Memory Networks (LSTM) is an optimized model of RNN. The LSTM model is mainly composed of a forget gate, an input gate, and an output gate. The input part includes C t-1 、h t-1 and x t , the internal structure of LSTM is as follows Figure 3 shown.

[0268] The role of forget gates is to determine the old state information C t-1 The output is a vector f with the same length as the cell state vector t , the value range is between [0,1]:

[0269] f t =σ*(W f [h t-1 ,x t ]+b f );

[0270] The role of the incoming gate is to determine h t-1 and x t What information is stored in the cell? This is mainly divided into two parts. The Sigmoid layer determines which values ​​will be updated, and the weighted tanh layer determines the candidate vectors added to the cell state.

[0271] i t =σ(W i [h t-1 ,x t ]+b i );

[0272]

[0273] By f t Expression of C t-1 The expected replacement part, i t Expressed new information The expected additions to update the status information:

[0274]

[0275] The output gate is used to determine the final output information h t , the Sigmoid layer passes through o t Determine the output and normalize the value to [-1, 1] using the tanh function:

[0276] o t =σ(W o [h t-1 ,x t ]+b o );

[0277] h t =o t *tanh(C t );

[0278] Since in the unidirectional LSTM network, the model processes time series data in a positive direction, C t-1 With h t-1 It contains historical information before time t, but does not learn data after time t. This one-way learning method increases the risk of model overfitting and reduces the accuracy of reconstructed data.

[0279] The bidirectional long short-term memory network (BiLSTM) combines an LSTM network that moves from the beginning to the end of the sequence and an LSTM network that moves from the end of the sequence to the beginning of the sequence. The output at time t comprehensively considers several historical input information and several future input information, such as Figure 4 shown.

[0280] In the forward layer, the input time series X = {X1, X2…, X i}The hidden state of each time step, get X i The forward output Then the Backward layer performs reverse calculation from time i to time 1 to obtain X i The reverse output of

[0281] The forward layer and the backward layer are connected to the output layer together, combining the output results of the two LSTM networks at each moment. and The final output is obtained by adding, averaging or concatenating.

[0282] 3. Variational Autoencoder (VAE)

[0283] A typical autoencoder (AE) is a three-layer neural network that uses the encoding layer to transform the original n-dimensional input data Mapped to the m-dimensional intermediate hidden layer, extracting nonlinear features The decoding layer then reconstructs the hidden layer vector into n-dimensional output data The training process is to minimize the input X i and output Y iThe reconstruction error is the optimization target of the model parameters, and the calculation formula of the reconstruction error loss is as follows:

[0284]

[0285] Traditional autoencoders are prone to overfitting. The overall structure of the variational autoencoder (VAE) is similar to that of the autoencoder AE. The difference is that the model perturbs the features extracted from the middle hidden layer to generate new samples that are similar but different from the training data. Figure 5 shown.

[0286] The encoding layer outputs two feature vectors m = {m1, m2..., m m} and σ={σ1,σ2...,σ m}, and then generate a vector data e={e1,e2...,e m}, finally execute the formula to get the final hidden layer feature vector C i =exp(σ i )×e i +m i .

[0287] In actual production environments, the time when abnormal situations occur during daily equipment operation is much shorter than the time when it operates normally. The frequency of abnormalities is very low, resulting in insufficient abnormal data available for analysis. In addition, because the equipment working scenarios of smart factories in the cloud-edge collaborative mode are constantly changing dynamically, business types are constantly updated and upgraded, resulting in a diversity of abnormal fault types. As an unsupervised learning algorithm, VAE only needs to be trained based on normal data samples, without learning the characteristics of fault data. By comparing the difference between the original data X and the reconstructed data Y, if the residual between the reconstructed data and the original data is too large (that is, the reconstruction error is too large), it is considered that there may be an abnormality.

[0288] Although VAE can solve the problem of sample size imbalance between normal data and abnormal data, it is difficult to identify the residual interval of normal data and abnormal data, and has poor learning effect on time series data. Therefore, the present invention embeds the Bi-LSTM network with better time series data processing effect into the encoding layer and decoding layer of VAE to construct a BiLSTM-VAE model, whose structure is as follows: Figure 6 shown.

[0289] BiLSTM-VAE model through X i and Y i The residual Loss i The abnormality is determined by comparing it with the reconstruction threshold δ. The calculation formula of the threshold is as follows:

[0290] σ=p·max(||XY|| 2 );

[0291]

[0292] Among them, δ is the reconstruction threshold, α and β represent the 90% quantile of the reconstruction error value set of the BiLSTM-VAE model for normal data and the 10% quantile of the reconstruction error value set of abnormal data, respectively.

[0293] The hyperparameters of the BiLSTM-VAE model are difficult to set. Inappropriate hyperparameters will lead to low model diagnostic accuracy and high false alarm rate. In addition, a single fault detection model has the problem that the reconstruction error intervals of normal data and abnormal data are not clearly differentiated. Therefore, the present invention sets K BiLSTM-VAE fault detection models with different hidden layer dimensions to jointly obtain residuals. Each fault detection model will generate a set of reconstructed data. Finally, the fault detection result of each time step data is the weighted sum of the reconstruction errors of each fault detection model under the weight matrix.

[0294] During the specific implementation process, the edge server performs weighted calculation on the reconstruction errors output by the fault detection models of each edge node to obtain a comprehensive reconstruction error; then performs weighted calculation based on the residual thresholds corresponding to each fault detection model to obtain a comprehensive residual threshold; finally, the comprehensive reconstruction error and the comprehensive residual threshold are compared to determine whether there is a fault in the device.

[0295] The comprehensive reconstruction error is calculated by the following formula:

[0296]

[0297] Where: Loss i represents the time series data X of K fault detection models for running data i The comprehensive reconstruction error of loss j,i represents the jth fault detection model k j For the time series data X of the running data i The reconstruction error of Denotes the fault detection model k j The reconstruction error weight of

[0298] The comprehensive residual threshold is calculated by the following formula:

[0299]

[0300] Where: j Denotes the fault detection model k j The residual threshold of Denotes the fault detection model k j The residual threshold weight of ;

[0301] When Loss l When <δ, the time series data X of the operation data i Normal, the device is normal; Loss l ≥δ, the time series data X of the running data i Abnormal. The device is abnormal.

[0302] The present invention uses the edge server to use the operating data as the input of each edge node fault detection model, and performs weighted calculation on the output of each edge node fault detection model to determine whether there is a fault in the equipment, so that the equipment abnormality detection performance can be improved through the integrated complementarity between multiple parallel edge nodes arranged on the edge server and the fault detection models with different hidden layer dimensions, that is, the fault detection result of each time step data is the weighted sum of the reconstruction errors of each fault detection model under the weight matrix, thereby further improving the accuracy of equipment abnormality detection.

[0303] At the same time, the fault detection model in the present invention (i.e., the BiLSTM-VAE model) can be directly solved by the whale optimization algorithm, without relying on manual parameter adjustment, without the need to build a model structure specific to the problem scenario, and without the need for prior experience and expertise, thereby further improving the practicality of equipment anomaly detection.

[0304] In the specific implementation process, Figure 7 As shown in the figure, the cloud trains the fault detection model through the following steps:

[0305] S201: Obtain a sample data set and perform data preprocessing, and then divide the preprocessed data into a training set consisting of all normal labeled data, a test set consisting of all abnormal labeled data, and a validation set consisting of a mixture of normal data and abnormal data;

[0306] S202: Setting K0 fault detection models with different hidden layer dimensions, and then using the training set to train each fault detection model to achieve feature learning of the normal state data of the equipment, and generate an initial residual threshold matrix;

[0307] S203: Eliminate K1 fault detection models with the worst performance to obtain K2=K0-K1 fault detection models;

[0308] S204: Initializing the reconstruction error weight matrix and the residual threshold weight matrix corresponding to the remaining fault detection models through a chaotic strategy;

[0309] In this embodiment, initializing the parameters of the deep model by using a chaos strategy is an existing conventional method and will not be described in detail here.

[0310] S205: The optimal reconstruction error weight matrix and residual threshold weight matrix are obtained by solving the Whale Optimization Algorithm (WOA), and then the fault detection model k is obtained. j and its corresponding reconstruction error weight Residual Threshold Weight and residual threshold σ j ;

[0311] S206: Test the performance of each fault detection model through the test set. If the performance meets the standard, end the training and verify the training effect of each fault detection model through the verification set; otherwise, return to step S204.

[0312] In the specific implementation process, data preprocessing includes Z-score standardization processing (using one-hot encoding technology to discretely digitize fault labels) and feature enhancement processing; when enhancing features, statistical features (first-order differences) and time series prediction features are extracted. Time series prediction features include the predicted value of the exponentially weighted moving average (EWMA) method and the difference between the predicted value and the original value;

[0313] Among them, the predicted value of the exponentially weighted moving average method is calculated by the following formula:

[0314]

[0315] Where: y i represents the predicted value of the exponentially weighted moving average method at time i; x i Represents the original value at time i; t represents the total step length of the time series data; the coefficient α represents the rate of weighted decrease. The larger its value is, the faster the weight of the early data decreases.

[0316] In the specific implementation process, the false alarm rate and missed alarm rate are used to evaluate each fault detection model: the smaller the false alarm rate and missed alarm rate, the better the effect of the fault detection model; the K1 fault detection models at the end of the comprehensive evaluation are eliminated, and the K2 = K0-K1 fault detection models with the highest comprehensive evaluation are selected;

[0317] in,

[0318]

[0319] Where: FAR represents the false alarm rate; MAR represents the missed alarm rate; TP represents the number of times the fault detection model accurately detects abnormal points; FP represents the number of times the fault detection model identifies normal points as abnormal points; FN represents the number of times the fault detection model identifies abnormal points as normal points; TN represents the number of times the fault detection model accurately identifies normal points.

[0320] In the specific implementation process, the performance of the fault detection model is evaluated by three evaluation indicators: precision, recall, and F-score;

[0321] in,

[0322]

[0323]

[0324] Where: Precision represents precision; TP represents the number of times the fault detection model accurately detects abnormal points; FP represents the number of times the fault detection model identifies normal points as abnormal points; Recall represents the recall rate; FN represents the number of times the fault detection model identifies abnormal points as normal points.

[0325] Since the training and debugging of deep learning models and swarm intelligence evolution algorithms (i.e., fault detection models) require a large amount of computing resources to ensure that the algorithm model achieves a high level of accuracy within a limited time, edge servers with limited computing power and storage space cannot be responsible for training tasks with large amounts of computing. Therefore, the present invention uses a cloud computing center equipped with a large amount of computing resources to train and update the model, thereby ensuring the fault detection accuracy of the fault detection model.

[0326] In order to verify the effectiveness of the cloud-edge collaborative anomaly detection system and the BiLSTM-VAE-WOA algorithm (i.e., the framework of the fault detection model + whale optimization algorithm) proposed in the present invention, the following experiments are disclosed in this embodiment.

[0327] This experiment is applied on the independently developed smart factory cloud-edge collaborative network management platform, such as Figure 8 shown.

[0328] 1. Experimental Data

[0329] The effectiveness and superiority of the proposed method are experimentally verified using laboratory equipment data and a server hard disk dataset. Since smart factories generate a large amount of equipment operation data during the production process, cloud-edge collaborative networks require a large number of solid-state drives (SSDs) to meet the storage needs of big data. The frequency of SSD failures will affect the stability and reliability of the server and even the entire cloud-edge collaborative network, and the security of the data cannot be protected. Therefore, real-time detection of server hard disk failures is one of the major issues of large-scale IT data facilities.

[0330] 2. Experimental parameters

[0331] After multiple adjustments to the algorithm parameters, it was finally found that the best anomaly detection effect was achieved when the parameters of the BiLSTM-VAE model and the WOA algorithm were set according to Table 4 and Table 5. The parameter settings of the BiLSTM-VAE model are shown in Table 4, where K0=30 is set as a fault detection model with different hidden layer dimensions; features represents the number of input features of the data set; the hidden layer dimension hidden_dim ranges from 10 to 160, with an interval of 5. When the WOA algorithm sets parameters according to Table 5, the optimization effect is the best.

[0332] Table 4 BiLSTM-VAE model parameters

[0333]

[0334] Table 5 WOA algorithm parameters

[0335]

[0336] 3. Experimental Results

[0337] 3.1 Experimental results

[0338] First, the reconstruction results of the data by the anomaly detection integrated framework and the single BiLSTM-VAE fault detection model are compared. The BiLSTM-VAE1 with the highest fit on the normal data set and the BiLSTM-VAE2 with the highest detection accuracy on the abnormal data set are selected respectively. The distribution of the reconstruction errors of the data by BiLSTM-VAE-WOA, BiLSTM-VAE1 and BiLSTM-VAE2 is shown in the figure below. Figure 8 , Fig. 9 , Fig.10 As shown in the figure, the points below the critical line represent normal data, and the points above the critical line represent abnormal data. The residual density distribution probability of normal data and abnormal data is as follows Fig.11 shown.

[0339] It can be seen that due to the different hidden layer dimensions, the reconstruction errors of BiLSTM-VAE1 and BiLSTM-VAE2 for abnormal data are quite different. The reconstruction error of the former is relatively large and dispersed, while the reconstruction error of the latter is smaller and more concentrated. The reconstruction error of BiLSTM-VAE-WOA for abnormal data is the most concentrated, and the distribution range of its reconstruction error density for normal sequences is also more to the left, and the overlapping area of ​​normal and abnormal parts is significantly smaller than that of a single fault detection model.

[0340] The results show that the single fault detection model is inferior to the BiLSTM-VAE-WOA method in all indicators, indicating that the anomaly detection integrated framework effectively optimizes the feature learning and fault detection performance of the single BiLSTM-VAE model and improves the overall detection accuracy of identifying abnormal state targets.

[0341] In order to further evaluate the proposed method, the anomaly detection ensemble framework is compared with representative algorithms on the market such as MSET, LSTM-AE, T-squared+Q (PCA), and LSTM, and their false alarm rate (FAR), false negative rate (MAR), and F1 indicators. The proposed BiLSTM-VAE-WOA method has the best performance in all the above indicators, which shows that the ensemble method is better at identifying abnormal sequences in time series than other methods, and has the highest detection accuracy.

[0342] In addition, in order to verify the performance of the proposed method, not only classic algorithms such as SDAE, PCA, KPCA, LGPCA and LSTM-AE were used for testing, but also some new improved deep learning models such as 1DRCAE algorithm, CLSTM-AE model, LE-DBN model and SSAE algorithm were cited for comparison.

[0343] The results show that the comparison methods are inefficient in learning feature information related to faults from time series data, and are unable to effectively detect these faults. However, the BiLSTM-VAE-WOA of the present invention can very effectively identify three types of faults, with a detection success rate of 100%, and performs well in detecting minor faults. The average FDR / DR index of the BiLSTM-VAE-WOA method for 21 faults is optimal. These test results demonstrate the strong feature learning and fault detection performance of the method proposed in this patent.

[0344] 3.2 Server Hard Drive SSD Dataset Experiment

[0345] The server hard disk SSD dataset comes from the operating data of more than 500,000 hard disks of the independently developed platform server. The dataset consists of original SMART data and standardized SMART data in 102 dimensions, which includes the operating status data of the hard disk hardware such as the head, disk, motor, and circuit.

[0346] Fig.12 The results of the BiLSTM-VAE-WOA algorithm detecting server hard disk failures are shown. The normal operation data and failure data are clearly differentiated, proving that the algorithm can better solve problems such as frequent data noise and extremely unbalanced ratios of normal data and abnormal data in the data set. Points below the critical line represent normal data, and points above the critical line represent abnormal data.

[0347] Depend on Fig.13 It can be seen that due to the difference in hidden layer dimensions, the residual calculated by BiLSTM-VAE1 is relatively small, and the residual calculated by BiLSTM-VAE2 is relatively large, but the overlap area between the normal and abnormal parts is large, so that the time series data with residuals near the abnormal threshold cannot be correctly judged. The integrated algorithm BiLSTM-VAE-WOA of the present invention has a residual density distribution interval for abnormal data that is further to the right, and a smaller overlap area makes the residual differentiation more obvious.

[0348] In summary, the present invention aims at the problem that equipment anomalies in smart factories are difficult to detect due to the variety of equipment and diversified fault types in the cloud-edge collaborative mode. It combines variational autoencoders, BiLSTM, WOA and other algorithms to propose a fault detection model (BiLSTM-VAE-WOA). The fault detection model first uses a training set in the cloud to train multiple BiLSTM-VAEs with different hidden layer dimensions, and then solves the optimal residual threshold and weight matrix of the fault detection model through the WOA algorithm and the test set. Finally, in the anomaly detection stage, the edge server downloads the trained fault detection model and the optimal residual threshold and weight matrix, obtains the output generated by each fault detection model through parallel computing, and performs weighted integration to obtain the final anomaly judgment result.

[0349] Simulation experiments were conducted using the SKAB dataset, TEP dataset, and server hard disk SSD anomaly dataset. The experimental results show that compared with traditional anomaly detection methods, the model of the present invention can effectively reduce the false alarm rate of anomaly detection, improve accuracy and real-time performance, and the detection model has the best overall performance in the three experimental datasets, with strong adaptability and versatility, which is conducive to promoting the large-scale application of this method in various scenarios under the cloud-edge collaborative mode. This method detects the abnormal reaction frequency of the equipment operation process and reminds relevant managers, helping enterprises to obtain longer response time to take appropriate measures to prevent greater losses to production and operation activities.

[0350] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit the technical solution. Those skilled in the art should understand that those modifications or equivalent substitutions of the technical solution of the present invention that do not depart from the purpose and scope of the technical solution should be included in the scope of the claims of the present invention.

Claims

1. The device anomaly detection system based on the cloud-edge collaboration mode is characterized by: include: The device layer is used to obtain the operation data of the device; In the cloud, several trained fault detection models are deployed; The cloud trains the fault detection model through the following steps: S201: Obtain a sample data set and perform data preprocessing, and then divide the preprocessed data into a training set consisting of all normal labeled data, a test set consisting of all abnormal labeled data, and a validation set consisting of a mixture of normal data and abnormal data; S202: Setting K0 fault detection models with different hidden layer dimensions, and then using the training set to train each fault detection model to achieve feature learning of the normal state data of the equipment, and generate an initial residual threshold matrix; S203: Eliminate K1 fault detection models with the worst performance to obtain K2=K0-K1 fault detection models; S204: Initializing the reconstruction error weight matrix and the residual threshold weight matrix corresponding to the remaining fault detection models through a chaotic strategy; S205: Using the whale optimization algorithm to solve the optimal reconstruction error weight matrix and residual threshold weight matrix, and then obtain the fault detection model k j and its corresponding reconstruction error weight Residual Threshold Weight and residual threshold σ j ; S206: Testing the performance of each fault detection model through the test set. If the performance meets the standard, the training is terminated and the training effect of each fault detection model is verified through the validation set; otherwise, returning to step S204; The fault detection model is a BiLSTM-VAE model obtained by embedding a bidirectional long short-term memory network into the encoding layer and decoding layer of a variational autoencoder. The hidden layer dimensions of each fault detection model are different. An edge server having a plurality of parallel edge nodes respectively used for downloading corresponding fault detection models from the cloud; The edge server uses the operating data as the input of each edge node fault detection model, and performs weighted calculation on the output of each edge node fault detection model to determine whether the device has a fault; When a device fails, the cloud analyzes the failure using real-time and historical operation data, generates decision information, and sends it to the edge server; The edge server sends the decision information to the device layer so that the device layer can perform corresponding actions.

2. The device anomaly detection system based on the cloud-edge collaboration mode according to claim 1, characterized in that: When there is no fault in the device, the edge server uploads the operation data to the cloud at a preset frequency as historical operation data; when there is a fault in the device, the edge server uploads the operation data containing the fault information to the cloud at a frequency higher than the preset frequency as real-time operation data.

3. A device anomaly detection method based on a cloud-edge collaborative mode, characterized in that: The device anomaly detection system based on the cloud-edge collaboration mode in claim 1 is implemented, specifically comprising the following steps: S1: The device layer obtains the operating data of the device; S2: Each edge node of the edge server downloads the trained fault detection model from the cloud; The fault detection model is a BiLSTM-VAE model obtained by embedding a bidirectional long short-term memory network into the encoding layer and decoding layer of a variational autoencoder. The hidden layer dimensions of each fault detection model are different. S3: The edge server uses the operating data as the input of each edge node fault detection model, and performs weighted calculation on the output of each edge node fault detection model to determine whether the device has a fault; S4: When a device fails, the cloud analyzes the failure based on real-time and historical operation data, generates decision information, and sends it to the edge server; S5: The edge server sends the decision information to the device layer so that the device layer can perform corresponding actions.

4. The device anomaly detection method based on the cloud-edge collaboration mode according to claim 3 is characterized in that: In step S201, data preprocessing includes Z-score normalization processing and feature enhancement processing; When the feature enhancement process is performed, statistical features and time series prediction features are extracted; the time series prediction features include the predicted value of the exponentially weighted moving average method, and the difference between the predicted value and the original value; Among them, the predicted value of the exponentially weighted moving average method is calculated by the following formula: Where: y i represents the predicted value of the exponentially weighted moving average method at time i; x i represents the original value at time i; t represents the total step length of the time series data; The coefficient α represents the rate of weighted decrease. The larger its value is, the faster the weight of early data decreases.

5. The device anomaly detection method based on the cloud-edge collaboration mode according to claim 3 is characterized in that: In step S203, each fault detection model is evaluated by using the false alarm rate and the missed alarm rate: the smaller the false alarm rate and the missed alarm rate, the better the effect of the fault detection model; the K1 fault detection models at the end of the comprehensive evaluation are eliminated, and the K2=K0-K1 fault detection models with the highest comprehensive evaluation are selected; in, Where: FAR represents the false alarm rate; MAR represents the missed alarm rate; TP represents the number of times the fault detection model accurately detects abnormal points; FP represents the number of times the fault detection model identifies normal points as abnormal points; FN represents the number of times the fault detection model identifies abnormal points as normal points; TN represents the number of times the fault detection model accurately identifies normal points.

6. The device anomaly detection method based on the cloud-edge collaboration mode according to claim 3 is characterized in that: In step S205, for the whale optimization algorithm: The original fixed probability value of the whale optimization algorithm is replaced by a dynamically changing AP value; AP=min_AP+max_AP*t / t max ; Where: t is the current number of iterations; t max Maximum number of iterations; The iterative formula of the whale's position in the stage of contraction and encirclement of prey is expressed as: C = 1.5*sin(2π*rand3)+0.5; Where: rand1, rand2 and rand3 are all random numbers between [0,1]; denote the positions of humpback whale i at iterations t and t+1, respectively; is an individual randomly selected from the current population; D represents the individual Iterative moving distance; A represents individual Iterative moving direction; a represents the nonlinear convergence factor; C is a random value in the range [0, 2]; β represents the adjustment of the collection speed; The iterative formula for the whale position in the hunting phase is expressed as: l = a2 + rand4; Where: is a randomly selected individual from the current elite solution set; They represent the position of humpback whale i at the tth and t+1th iterations respectively; rand4 is a random number between [0,1]; b is a constant that limits the shape and size of the spiral; l is improved from a random number between [-1,1] to a coefficient that decays nonlinearly with the algorithm iteration process, which is used to control the shape of the logarithmic spiral; a2 represents the nonlinear convergence factor; β represents the adjustment of the collection speed; The search formula for new prey is expressed as: Where: represents the position of humpback whale i at the t+1th iteration; is a randomly selected individual from the current elite solution set; μ and v follow the standard normal distribution: A represents the displacement step; Levy (γ) represents the Levy flight function; γ is the coefficient constant; a3 represents the nonlinear convergence factor; φ represents the step size of the Levy flight; Γ represents the gamma function; exp represents exponential operation; β represents the adjustment of the collection speed.

7. The device anomaly detection method based on the cloud-edge collaboration mode according to claim 3 is characterized in that: In step S206, the performance of the fault detection model is evaluated by three evaluation indicators: precision, recall and F-score; in, Where: Precision represents precision; TP represents the number of times the fault detection model accurately detects abnormal points; FP represents the number of times the fault detection model identifies normal points as abnormal points; Recall represents the recall rate; FN represents the number of times the fault detection model identifies abnormal points as normal points.

8. The device anomaly detection method based on the cloud-edge collaboration mode according to claim 3 is characterized in that: In step S3, the edge server performs weighted calculation on the reconstruction errors output by the fault detection models of each edge node to obtain a comprehensive reconstruction error; then performs weighted calculation based on the residual thresholds corresponding to each fault detection model to obtain a comprehensive residual threshold; finally, the comprehensive reconstruction error and the comprehensive residual threshold are compared to determine whether the device has a fault.

9. The device anomaly detection method based on the cloud-edge collaboration mode according to claim 8, characterized in that: The comprehensive reconstruction error is calculated by the following formula: Where: Loss i represents the time series data X of K fault detection models for running data i The comprehensive reconstruction error of loss j,i represents the jth fault detection model k j For the time series data X of the running data i The reconstruction error of Denotes the fault detection model k j The reconstruction error weight of The comprehensive residual threshold is calculated by the following formula: Where: j Denotes the fault detection model k j The residual threshold of Denotes the fault detection model k j The residual threshold weight of ; When Loss i When <δ, the time series data X of the operation data i Normal, there is no abnormality in the device; Loss i ≥δ, the time series data X of the operating data i Abnormal. The device is abnormal.

Images