A method for detecting time series change points of multi-sensor information based on fusion confidence strategy

Abstract

The present invention discloses a method for detecting time series change points of multivariate sensor information with a fusion confidence strategy, comprising: obtaining time series information of the multivariate sensor to be detected; performing format preprocessing on the time series information of the multivariate sensor to obtain three groups of sample vectors: time series multivariate sample vectors, differential multivariate sample vectors and frequency domain multivariate sample vectors; inputting the sample vectors into a pre-trained neural network model to obtain a prediction result output by the model; the neural network model comprises a time series sub-neural network model, a differential sub-neural network model, a frequency domain sub-neural network model and a fusion sub-neural network model; and determining whether the device state at the time corresponding to the multivariate sensor time series information has changed based on the detection result.

Description

Technical Field

[0001] The present invention relates to a method for detecting time series change points of multivariate sensor information integrated with a confidence strategy, and belongs to the technical field of time series data analysis and industrial sensor equipment monitoring applications. Background Art

[0002] With the rapid development of the Internet, new-generation information technology represented by the Internet of Things, big data and artificial intelligence is accelerating its integration with traditional industries. A new round of global scientific and technological revolution and industrial transformation is booming, and new production methods, organizational methods and business models are constantly emerging, promoting the intelligent transformation of the global industrial system.

[0003] Today's industrial equipment is vast, and the connections between them are complex. This makes equipment maintenance and manual troubleshooting very expensive. Furthermore, the growing need for safe production demands increases the need for accurate detection of equipment anomalies. Therefore, designing an efficient, accurate, stable, and robust method for detecting equipment anomalies is becoming increasingly important.

[0004] Typically, the primary basis for detecting device status anomalies comes from sensors embedded in the device, as well as sensors deployed externally to monitor overall system resource availability. Data from these sensors can be considered a form of time-series sensor information. Therefore, the specific task of detecting device status anomalies by analyzing various time-series sensor information can be abstracted into the broader task of detecting time-series change points in multivariate time-series sensor information.

[0005] Change point detection is a specialized part of the time series data problem that aims to detect anomalies in a collection. Closely related to the well-known change point estimation or change point mining problem, change point detection is the problem of finding sudden changes in the data when the attributes of a time series change.

[0006] Previously, most of the time series change point detection focused on a single element, that is, a single time series sensor information, while the feature capture capability of multi-element time series sensor information was relatively weak.

[0007] In addition, past change point detection methods focused on improving the accuracy of the results or the real-time performance of the methods, but few people mentioned research on the reliability and credibility of the decision.

[0008] In addition, in actual application scenarios, it is difficult to obtain a large amount of carefully labeled data. Instead, there is often a large amount of unlabeled data available, so self-supervised learning methods are very necessary.

[0009] In summary, the current research on change point detection of multi-element time series sensor information still has the following shortcomings: First, the feature capture capability is insufficient in the face of multi-element time series sensor information; Second, it is impossible to explain the output decision well based on the multi-element time series sensor input and provide effective guidance to users; Third, the deep learning-based algorithm requires a high amount of carefully labeled data, which is costly to meet. Summary of the Invention

[0010] The purpose of the present invention is to provide a method for detecting time series change points of multi-sensor information that integrates a confidence strategy, so as to solve the problems in the prior art such as insufficient feature capture capability, inability to well explain output decisions, and the requirement for a complete and large amount of labeled data sets to complete model pre-training.

[0011] To achieve the above objectives, the present invention is implemented by adopting the following technical solutions:

[0012] In a first aspect, the present invention provides a method for detecting time series change points of multi-sensor information by integrating a confidence strategy, comprising:

[0013] Obtaining timing information of the multi-sensor to be detected;

[0014] The multivariate sensor time series information is format preprocessed to obtain three groups of sample vectors: time series multivariate sample vectors, differential multivariate sample vectors and frequency domain multivariate sample vectors;

[0015] Inputting the sample vector into a pre-trained neural network model to obtain a prediction result output by the model; wherein the neural network model includes a time series sub-neural network model, a differential sub-neural network model, a frequency domain sub-neural network model and a fusion sub-neural network model; inputting the time series multivariate sample vector into the time series sub-neural network model to extract a threshold feature, inputting the differential multivariate sample vector into the differential sub-neural network model to extract a change feature, inputting the frequency domain multivariate sample vector into the frequency domain sub-neural network model to extract a frequency domain feature, inputting the threshold feature, the change feature and the frequency domain feature into the fusion sub-neural network model for fusion to obtain a detection result;

[0016] According to the detection result, it is determined whether the device state corresponding to the time of the multi-sensor timing information changes.

[0017] In some embodiments, the multi-sensor time series information is format preprocessed to obtain three groups of sample vectors, including:

[0018] Normalize the multivariate sensor time series information according to its maximum value and then perform sliding window cropping to obtain a time series multivariate sample vector;

[0019] Perform self-differentiation on the time series multivariate sample vector to obtain a differential multivariate sample vector;

[0020] Perform short-time Fourier transform on the time series multivariate sample vector to obtain a frequency domain multivariate sample vector.

[0021] Furthermore, in some embodiments, the time series multivariate sample vector is self-differentiated, including: for a time series multivariate sample vector of length n, taking the first n-1 groups of time series multivariate sample vectors v1 and the last n-1 groups of time series multivariate sample vectors v2, and the result obtained by the self-differentiation method is a differential multivariate sample vector v3 of length n-1 obtained by v2-v1.

[0022] In some embodiments, the sequential sub-neural network model and the differential sub-neural network model adopt a bidirectional LSTM neural network;

[0023] The frequency domain sub-neural network model adopts a two-layer interactive transformer network;

[0024] The fusion sub-neural network model adopts a multi-layer fully connected structure.

[0025] In some embodiments, the training method of the neural network model includes:

[0026] Acquire a training data set for format preprocessing, wherein the training data set includes labeled training data and unlabeled training data;

[0027] Use labeled training data to input into the neural network model for labeled training;

[0028] Use unlabeled training data to input into the neural network model for self-distillation training;

[0029] Label training and self-distillation training are carried out interactively, and the cycle is iterated until the preset conditions are met to obtain a trained neural network model.

[0030] In some embodiments, the labeled training uses subjective logistic loss L(Θ):

[0031]

[0032] Among them, Θ is the parameter of the neural network, N is the total number of samples in the current batch, k means that the kth sample is selected from the current batch for calculation, and λ epoch is the annealing coefficient, expressed as epoch is the index of the current training round, is the early attenuation coefficient; L k (Θ) is the Bayesian risk, KL[Be(p|α p ,α n )||Be(p|1,1)] is the divergence loss;

[0033] Bayesian risk Lk The calculation formula for (Θ) is:

[0034]

[0035] where ψ(·) is the digamma function; k represents the kth sample, y k ={y kp ,y kn} is a two-dimensional one-hot vector, indicating whether the actual category of the k-th sample is positive or negative. Positive refers to the change of the device under test, corresponding to y k ={0,1}, negative refers to the device under test does not change, corresponding to y k ={1,0};α k ={α kp ,α kn} is the Beta distribution density parameter on the predictor variable, p k ={p kp ,p kn} is the probability of being predicted as a change point or a non-change point; Be(p k |α k ) is regarded as the likelihood B(y k |p k ), Be(·) is the Beta distribution function, and B(·) is the binomial distribution function;

[0036] Divergence loss KL[Be(p|α p ,α n )||Be(p|1,1)] is calculated as:

[0037]

[0038] in is the prediction parameter α from sample k k The Beta parameter after removing non-misleading evidence, Be(p|a,b) corresponds to the Beta distribution with parameters a and b in statistics; is the result after removing non-misleading evidence from the evidence that the kth sample in the batch is a changed sample, e k It is the neural network judgment evidence, corresponding to the intermediate output of the neural network.

[0039] In some embodiments, the self-distillation training uses a multi-model mutual KL loss:

[0040] L(e t ,e d ,e f )=D KL ″(e t ||e d )+D KL ″(ed ||e f )+D KL ″(e f ||e t )

[0041] Where L(e t ,e d ,e f ) is the multi-model mutual KL loss, D KL ″(α||b) is the exponential self-KL divergence loss, e t ,e d ,e f They refer to the features extracted from samples by the time-series sub-neural network, the difference sub-neural network, and the frequency-domain sub-neural network, respectively, and correspond to the intermediate outputs of the sub-neural networks after weighting and before summation;

[0042]

[0043] Where a and b refer to sub-model a and sub-model b respectively, a(x i ) and b(x i ) refer to the amount of evidence output by sub-model a and sub-model b for category i, i∈{p,n} represents the category judged as changed and the category judged as unchanged, e (·) Indicates taking the natural exponent.

[0044] In some embodiments, the self-distillation training refers to fixing the parameters of a neural network model, called the Teacher Model, and updating the parameters of other neural network models, called the Student Model, within a certain period.

[0045] In a second aspect, the present invention provides a device for detecting time series change points of multi-sensor information integrated with a confidence strategy, comprising a processor and a storage medium;

[0046] The storage medium is used to store instructions;

[0047] The processor is configured to operate according to the instructions to execute the steps of the method according to the first aspect.

[0048] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processor.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] This invention innovatively adopts the idea of ​​integrated multimodality. By preprocessing the time series sequence, three features, namely time series, differential and frequency domain, are generated. A neural network is designed to deeply extract each preprocessed feature, and finally fuse and detect them. For the first time, the concept of uncertainty is introduced in the change point detection task. Through the DST deep learning method, the concept of belief quality is introduced, so that the model gives the uncertainty of the judgment while outputting the detection results and probabilities for the user's reference. On the basis of the subjective logical loss combined with the divergence, the imbalance coefficient is innovatively introduced to deal with samples with extremely unbalanced distribution. In order to make full use of unlabeled samples, semi-supervised training is introduced, that is, the self-supervised learning method is adopted at a certain stage, and combined with the idea of ​​self-knowledge distillation, a multi-model self-KL loss is proposed to obtain higher accuracy and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a flowchart of a method for detecting time series change points of multi-sensor information integrated with a confidence strategy provided by an embodiment of the present invention;

[0052] Figure 2 This is a pre-training flow chart of a method for detecting time series change points of multi-sensor information integrated with a confidence strategy provided by an embodiment of the present invention;

[0053] Figure 3 Schematic diagram of the overall structure of a neural network provided by an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of the structure of the sequential sub-neural network and the differential sub-neural network of the neural network provided in an embodiment of the present invention, wherein the left side is a schematic diagram of the LSTM and the right side is a schematic diagram of the structure common to the above two neural networks;

[0055] Figure 5 Schematic diagram of a frequency domain sub-neural network of a neural network provided by an embodiment of the present invention.

[0056] Figure 6 This is a flowchart of multi-model mutual KL divergence loss calculation and back propagation provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0058] Example 1

[0059] A method for detecting time series change points of multi-sensor information integrated with a confidence strategy, comprising:

[0060] Obtaining timing information of the multi-sensor to be detected;

[0061] The multivariate sensor time series information is format preprocessed to obtain three groups of sample vectors: time series multivariate sample vectors, differential multivariate sample vectors and frequency domain multivariate sample vectors;

[0062] Inputting the sample vector into a pre-trained neural network model to obtain a prediction result output by the model; wherein the neural network model includes a time series sub-neural network model, a differential sub-neural network model, a frequency domain sub-neural network model and a fusion sub-neural network model; inputting the time series multivariate sample vector into the time series sub-neural network model to extract a threshold feature, inputting the differential multivariate sample vector into the differential sub-neural network model to extract a change feature, inputting the frequency domain multivariate sample vector into the frequency domain sub-neural network model to extract a frequency domain feature, inputting the threshold feature, the change feature and the frequency domain feature into the fusion sub-neural network model for fusion to obtain a detection result;

[0063] According to the detection result, it is determined whether the device state corresponding to the time of the multi-sensor timing information changes.

[0064] Timing change points can generally be divided into three categories:

[0065] The first type is threshold change, that is, there are data that is too high or too low within a certain period of time. Usually, neural networks have a strong ability to capture such change characteristics, so there is no need to transform the original time series.

[0066] The second category is change change, that is, within a certain period of time, the change of time series values ​​is particularly drastic, or changes that do not normally occur occur. This type of change feature can often be detected by neural networks, but its effect is not as obvious as the first category. By performing differential operations on time series data and extracting features from the differential data, this type of change is often easily extracted.

[0067] The third type is disturbance change, which occurs when high-frequency or low-frequency disturbances occur within a certain period of time that should not exist. In frequency-sensitive scenarios, this type of disturbance is often fatal. A more effective method is the FFT, which obtains the frequency domain features of time series data and detects disturbance changes through in-depth extraction of these features.

[0068] In some embodiments, for the three aforementioned change categories, format preprocessing is performed on the multi-sensor time series information to obtain three groups of sample vectors, including:

[0069] Normalize the multivariate sensor time series information according to its maximum value and then perform sliding window cropping to obtain a time series multivariate sample vector;

[0070] Perform self-differentiation on the time series multivariate sample vector to obtain a differential multivariate sample vector;

[0071] Perform short-time Fourier transform on the time series multivariate sample vector to obtain a frequency domain multivariate sample vector.

[0072] Furthermore, in some embodiments, the time series multivariate sample vector is self-differentiated, including: for a time series multivariate sample vector of length n, taking the first n-1 groups of time series multivariate sample vectors v1 and the last n-1 groups of time series multivariate sample vectors v2, and the result obtained by the self-differentiation method is a differential multivariate sample vector v3 of length n-1 obtained by v2-v1.

[0073] In some embodiments, the sequential sub-neural network model and the differential sub-neural network model adopt a bidirectional LSTM neural network;

[0074] The frequency domain sub-neural network model adopts a two-layer interactive transformer network;

[0075] The fusion sub-neural network model adopts a fully connected layer structure.

[0076] In some embodiments, the training method of the neural network model includes:

[0077] Acquire a training data set for format preprocessing, wherein the training data set includes labeled training data and unlabeled training data;

[0078] Use labeled training data to input into the neural network model for labeled training;

[0079] Use unlabeled training data to input into the neural network model for self-distillation training;

[0080] Label training and self-distillation training are carried out interactively, and the cycle is iterated until the preset conditions are met to obtain a trained neural network model.

[0081] Among them, the labeled training adopts subjective logistic loss L(Θ):

[0082]

[0083] Among them, Θ is the parameter of the neural network, N is the total number of samples in the current batch, k means that the kth sample is selected from the current batch for calculation, and λ epoch is the annealing coefficient, expressed as epoch is the index of the current training round, is the early attenuation coefficient; L k (Θ) is the Bayesian risk, KL[Be(p|α p ,α n )||Be(p|1,1)] is the divergence loss;

[0084] Bayesian risk L k The calculation formula for (Θ) is:

[0085]

[0086] where ψ(·) is the digamma function; k represents the kth sample, y k ={y kp ,y kn} is a two-dimensional one-hot vector, indicating whether the actual category of the k-th sample is positive or negative. Positive refers to the change of the device under test, corresponding to y k ={O,1}, negative means that the device under test does not change, corresponding to y k ={1,0};α k ={α kp ,α kn} is the Beta distribution density parameter on the predictor variable, p k ={p kp ,p kn} is the probability of being predicted as a change point or a non-change point; Be(p k |α k ) is regarded as the likelihood B(y k |p k ), Be(·) is the Beta distribution function, and B(·) is the binomial distribution function;

[0087] Divergence loss KL[Be(p|α p ,α n )||Be(p|1,1)] is calculated as:

[0088]

[0089] is the prediction parameter α from sample k k The Beta parameter after removing non-misleading evidence, Be(p|a,b) corresponds to the Beta distribution with parameters a and b in statistics; is the result after removing non-misleading evidence from the evidence that the kth sample in the batch is a changed sample, e k It is the neural network judgment evidence, corresponding to the intermediate output of the neural network.

[0090] The self-distillation training adopts the multi-model mutual KL loss L(e t ,e d ,e f ):

[0091] L(e t ,e d ,e f )=DKL ″(e t ||e d )+D KL ″(e d ||e f )+D KL ″(e f ||e t )

[0092] Where L(e t ,e d ,e f ) is the multi-model mutual KL loss, D KL ″(e t ||e d ) is the exponential self-KL divergence loss, e t ,e d ,e f They refer to the features extracted from samples by the time-series sub-neural network, the difference sub-neural network, and the frequency-domain sub-neural network, respectively, and correspond to the intermediate outputs of the sub-neural networks after weighting and before summation;

[0093]

[0094] Where a and b refer to sub-model a and sub-model b respectively, a(x i ) and b(x i ) refer to the amount of evidence output by sub-model a and sub-model b for category i, i∈{p,n} represents the category judged as changed and the category judged as unchanged, e (·) Indicates taking the natural exponent. For the specific process, see Figure 6 .

[0095] The self-distillation training refers to fixing the parameters of a neural network model, called Teacher Model, and updating the parameters of other neural network models, called Student Model, within a certain period.

[0096] In some specific embodiments, a method for detecting time series change points of multi-sensor information integrated with a confidence strategy includes:

[0097] Obtaining the time series information of the multi-sensor to be detected, and performing format preprocessing on the time series information of the multi-sensor to obtain three groups of sample vectors;

[0098] Input the three sets of sample vectors into the pre-trained neural network to obtain the prediction vector;

[0099] The prediction vector contains multiple prediction values, each of which contains its own classification label, classification evidence, and classification uncertainty. The classification label with the largest prediction value is selected as the final classification label. If the final classification label is changed, the device state corresponding to the multivariate sensor timing information has changed, otherwise it has not changed. Classification uncertainty provides guidance for users to make judgments, that is, when the classification uncertainty is high, the user should combine their own judgment and conduct further analysis based on the judgment obtained by the present invention. When the classification uncertainty is low, the present invention maintains a high degree of confidence in its judgment, and the user does not need to modify the result based on their own judgment. For the usage process, please refer to the attached Figure 1 , the pre-training process is shown in the attached Figure 2 .

[0100] In combination with the first aspect, further, the neural network includes a time domain neural network, a differential neural network, and a frequency domain neural network. The overall structure is shown in the attached Figure 3 .

[0101] Furthermore, the time domain neural network and the differential sub-neural network use a bidirectional LSTM neural network, and the bidirectional features are fused through the merging layer, and then the original evidence of the sub-neural network is calculated through the evidence calculation layer. Figure 4 , where LSTML refers to the LSTM model with the same input order of sample vectors as the actual time sequence, and LSTMR refers to the LSTMR model with the same input order of sample vectors as the actual time sequence. The difference between the two is only reflected in the different input order of sample vectors. The model framework is the LSTM model. Figure 4 The left side shows the internal framework of the LSTM model, where tanh corresponds to the hyperbolic tangent function.

[0102] Furthermore, the frequency domain neural network is based on a two-layer interactive transformer network, which is interactively superimposed and bidirectionally integrated through a merging layer, and then the original evidence of the sub-neural network is calculated through an evidence calculation layer. Figure 5 , where the encoder and decoder layers are the same as those of a typical transformer network. Q, K, and V correspond to the query, key, and value of the encoder and decoder layers, respectively.

[0103] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting time series change points of multi-sensor information by integrating a confidence strategy, comprising the following steps:

[0104] S1. Acquire the time series information of the multi-sensor to be detected, and perform format preprocessing on the time series information of the multi-sensor to obtain a sample vector.

[0105] S2. Input the sample vector into the pre-trained neural network model to obtain the prediction vector.

[0106] Neural network models are pre-built, such as Figure 3 As shown in Figure 1, the neural network model is divided into four parts. The first three network structures consider time series, differential, and frequency domain characteristics, respectively, and perform feature extraction based on threshold, transformation, and perturbation changes. In the fusion part, the features obtained from the first three parts are fed into the weighted fusion classification layer.

[0107] In the first part of the model, the time series network is based on a stacked bidirectional recurrent neural network to capture the timing characteristics of network flows. See the attached diagram for a schematic diagram of the model structure. Figure 4 .

[0108] The second part of the model obtains the sequence change characteristics through differential processing. The change characteristics can supplement the features automatically extracted by deep learning from another perspective. The difference network used in this invention is also based on a superimposed bidirectional recurrent neural network to learn the specific patterns of change characteristics in time series. See the attached diagram for a schematic diagram of the model structure. Figure 4 .

[0109] The third part of the model considers frequency domain features, so there is no time series. From this perspective, frequency domain features are a discrete feature that is more difficult to extract than the first two. The frequency domain network is based on a two-layer interactive transformer network, which learns the frequency domain features of the time series through interactive superposition. See the attached diagram for the model structure. Figure 5 .

[0110] After fusion, the model outputs evidence for each category, that is, whether it is judged as a change or no change. The probability and uncertainty of the neural network's judgment can be calculated through the evidence.

[0111] Subjective Logic (SL) formalizes the concept of confidence distribution of DST in a discriminative framework as a Beta distribution. Therefore, it is possible to quantify confidence quality and uncertainty using probabilistic methods through a well-defined theoretical framework. More specifically, SL considers a framework consisting of K mutually exclusive class labels: by providing a confidence quality b for each class label and an overall uncertainty quality u. For the present invention, the present invention is a binary classification problem, that is, K=2, so the above framework can be simplified to the formula:

[0112] u+b p +b n =1

[0113] where u≥0 and bp ,b n ≥0. Where p,n represent positive samples and negative categories, i.e., predicted as change and no change. For the kth sample in the batch, the confidence quality b of the class label is calculated using the evidence of the class label k . Let e k ≥0 is the evidence derived from the i-th (i∈{p,n}) class label, then b i and u are calculated as:

[0114]

[0115] Where S = e p +e n The +2 represents the overall confidence quality, i.e., subjective opinion. It is worth noting that uncertainty is inversely proportional to the total evidence. When there is no evidence, the confidence of each single case is zero and the uncertainty is 1.

[0116] Confidence quality assignment, i.e. subjective opinion, is associated with parameter α i =e i +1 Beta distribution, we can use the above b i =e i / S=(α i -1) / S directly derives the subjective opinion S from the parameters of the corresponding Beta distribution, that is, S = α p +α n .

[0117] The method of representing the density of each such probability distribution using a Beta distribution parameterized by the evidence can model second-order probabilities and uncertainties.

[0118] The probability density function of the Beta distribution is given by:

[0119]

[0120] where p p ,p n Represent the probability of the sample being expected to be assigned as positive and negative, respectively, p n =1-p p B(α p ,α n ) is the Beta function.

[0121] By discarding Softmax and using ReLU activation layer as the model output constraint, we use the positive evidence derived during model training to generate a prior distribution for the multi-sample Beta distribution, thereby forming an opinion for the classification task. For a given sample k, let e k =f(x k|Θ) represents the evidence vector for the network classification prediction, where Θ is the network parameter and f(x|Θ) is the model output. Subsequently, the corresponding Beta distribution parameter α k It can be expressed as α k =f(x k |Θ)+1. Once the parameters of this distribution are calculated, its mean, α k / S k , can be used as an estimate of the class probability, that is, for the expected distribution probability p p ,p n , its calculation formula can be expressed as:

[0122]

[0123] S3. In the set of labeled training samples, the subjective logistic loss is calculated based on the predicted vector and the true label of the training sample. Let y k ={y kp ,y kn} is the true label of a sample in the labeled training sample set, which is a two-dimensional one-hot vector indicating whether the actual category of the k-th sample is positive or negative. Positive refers to the change, corresponding to y k ={0,1}, negative refers to no change corresponding to y k ={1,0}. p k ={p kp ,p kn} is the probability of being predicted as a change point and a non-change point. k ={α p ,α n} is the Beta distribution density parameter on the predictor variable. First, define a loss function and calculate its Bayesian risk relative to the class predictor. For cross entropy loss, the Bayesian risk is:

[0124]

[0125] Where ψ(·) is the digamma function. k represents the kth sample, y k ={y kp ,y kn} is a two-dimensional one-hot vector, indicating whether the actual category of the k-th sample is positive or negative. Positive refers to the change of the device under test, corresponding to y k ={0,1}, negative refers to the device under test does not change, corresponding to y k ={1,0}. α k ={α kp ,α kn} is the Beta distribution density parameter on the predictor variable, p k ={p kp ,p kn} is the probability of being predicted as a change point or a non-change point. k |α k ) is regarded as the likelihood B(y k |p k ), Be(·) is the Beta distribution function, and B(·) is the binomial distribution function

[0126] The loss for a batch of training samples can be calculated by summing the losses for each sample in the batch. During training, the model may discover patterns in the sample set and generate evidence for specific class labels based on these patterns to minimize the overall loss. When counterexamples to the criterion are observed during training, the parameters of the neural network should be adjusted through backpropagation so that a smaller amount of evidence is generated for the pattern, and the loss of these samples is minimized as the overall loss decreases. However, when the number of counterexamples is limited, reducing the size of the generated evidence will reduce the loss of the counterexamples, but may increase the overall loss. As a result, the neural network will generate some evidence for incorrect labels. Although as long as the sample can be correctly classified by the network, the evidence for the correct class label is higher than the evidence for other class labels. The divergence term is incorporated into the loss function of the subjective logic loss mentioned in this invention to regularize the prediction distribution by penalizing errors that are significantly different from highly uncertain states but do not contribute to data fitting. The loss of this regularization term is:

[0127]

[0128] Where Be(p|1,1) represents uniform Beta distribution, is the prediction parameter α from sample k k The Beta parameter after removing non-misleading evidence in . The divergence term in the loss can be calculated as:

[0129]

[0130] where Γ(·) is the gamma function, ψ(·) is the digamma function, is the result after removing non-misleading evidence from the evidence that the kth sample in the batch is a changed sample, e k It is the neural network judgment evidence, corresponding to the intermediate output of the neural network.

[0131] λ epoch is the annealing coefficient, expressed as epoch is the index of the current training round, is the early attenuation coefficient, when When λ epoch <1, as the epoch increases, that is, the number of model training increases, λ epochGradually increases and eventually converges to 1, through the annealing coefficient λ epoch Gradually increasing the influence of KL divergence in the loss can, to a certain extent, avoid premature convergence of misclassified samples to a uniform distribution.

[0132] Time series classification has attracted significant interest over the past decade. However, current research assumes the availability of large amounts of labeled training data. In practice, obtaining such data can be difficult or expensive. As in many other fields, large amounts of unlabeled data are often available.

[0133] One of the goals of the present invention is to make the best use of these unlabeled data so as to make the neural network model more accurate and more robust.

[0134] Knowledge distillation, a relatively new semi-supervised learning method, has achieved strong results in many tasks. It trains a comprehensive and large Teacher model. The Teacher model's output is used as soft labels to guide the learning of the Student model on unlabeled samples. This helps accelerate the learning of the Student model by inducing the output of the Student model to converge with the Teacher model.

[0135] This paper uses a loss function calculation method based on KL divergence loss. In statistical applications, it is often necessary to use a simple, approximate probability distribution f* to describe the observation data D or another complex probability distribution f. KL divergence loss is a method used to measure the amount of information lost by the approximate distribution f* compared to the original distribution f.

[0136] The information entropy corresponding to a binary probability distribution f is:

[0137] H f =-p(x p )·log(p(x p ))-p(x n )·log(p(x n ))

[0138] Among them, p(x i ) is the probability of being distributed in the i-th category, i∈{p,n}. The KL divergence calculation formula is a variation of the entropy calculation formula. The approximate probability distribution q is added to the original probability distribution p, and the expected value of the logarithmic difference between the original distribution of the sample and the predicted approximate probability distribution is calculated.

[0139]

[0140] Under normal circumstances, the probability distribution of the neural network output is generated by the Softmax function, and the predicted approximate distribution of the sample ranges from (0, 1). In other words, under normal circumstances, the predicted approximate distribution of the sample will not take extreme values ​​such as 0 or 1, so the calculation process of the KL divergence loss will not overflow. However, in this invention, since the model output is the label-derived evidence e, its distribution is [0, +∞], there is a risk of underflow. Therefore, the natural exponential is used for the KL divergence loss:

[0141]

[0142] This avoids underflow.

[0143] In addition, since the KL divergence loss used in self-supervised learning does not have an original distribution p, that is, the evidence distributions derived from the two input labels in the KL divergence loss contribute equally to the loss, the present invention uses the evidence distributions derived from the two input labels as the sample original distributions, calculates the KL divergence loss separately, and sums them. The above operation can be expressed as:

[0144]

[0145] It is worth noting that the present invention has three sub-neural network models, namely the time series model M for processing time series data t , used to process differential data differential model M d , and the frequency domain model M used to process frequency domain data f Each model corresponds to a set of label-derived evidence distributions. In order to avoid the loss overflow caused by the large gap in label-derived evidence between different sub-models, the present invention uses a maximum normalization operation on all evidences, that is, in the self-supervised learning process, the three types of group label-derived evidence distributions obtained by the three sub-neural network models are output as e t , e d , e f , applying this loss function to the model of the present invention, the total loss is:

[0146] L(e t ,e d ,e f )=D KL ″(e t ||e d )+D KL ″(e d ||e f )+D KL ″(e f ||e t )

[0147] This loss can be used to guide model learning so that the output of the model converges to a uniform distribution. Figure 6 .

[0148] Example 2

[0149] In a second aspect, this embodiment provides a device for detecting time series change points of multi-sensor information integrated with a confidence strategy, comprising a processor and a storage medium;

[0150] The storage medium is used to store instructions;

[0151] The processor is configured to operate according to the instructions to execute the steps of the method according to embodiment 1.

[0152] Example 3

[0153] In a third aspect, this embodiment provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in Example 1 are implemented.

[0154] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0155] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1The function specified in one or more boxes.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0158] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting time series change points of multi-sensor information by integrating confidence strategy, characterized in that: include: Obtaining timing information of the multi-sensor to be detected; The multivariate sensor time series information is format preprocessed to obtain three groups of sample vectors: time series multivariate sample vectors, differential multivariate sample vectors and frequency domain multivariate sample vectors; Inputting the sample vector into a pre-trained neural network model to obtain a prediction result output by the model; wherein the neural network model includes a time series sub-neural network model, a differential sub-neural network model, a frequency domain sub-neural network model and a fusion sub-neural network model; inputting the time series multivariate sample vector into the time series sub-neural network model to extract a threshold feature, inputting the differential multivariate sample vector into the differential sub-neural network model to extract a change feature, inputting the frequency domain multivariate sample vector into the frequency domain sub-neural network model to extract a frequency domain feature, inputting the threshold feature, the change feature and the frequency domain feature into the fusion sub-neural network model for fusion to obtain a detection result; Determining, based on the detection results, whether a device state corresponding to the time of the multi-sensor timing information has changed; The training method of the neural network model includes: obtaining a training data set that has been formatted, wherein the training data set includes labeled training data and unlabeled training data; using the labeled training data to input into the neural network model for labeled training; using the unlabeled training data to input into the neural network model for self-distillation training; the labeled training and self-distillation training are performed interactively and iteratively until a preset condition is met to obtain a trained neural network model; the labeled training uses a subjective logistic loss L(Θ): Among them, Θ is the parameter of the neural network, N is the total number of samples in the current batch, k means that the kth sample is selected from the current batch for calculation, and λ epoch is the annealing coefficient, expressed as epoch is the index of the current training round, is the early attenuation coefficient; L k (Θ) is the Bayesian risk, KL[Be(p|α p ,α n )||Be(p|1,1)] is the divergence loss.

2. The method for detecting time series change points of multi-sensor information using a fusion confidence strategy according to claim 1 is characterized in that: The multi-sensor time series information is formatted and preprocessed to obtain three groups of sample vectors, including: Normalize the multivariate sensor time series information according to its maximum value and then perform sliding window cropping to obtain a time series multivariate sample vector; Perform self-differentiation on the time series multivariate sample vector to obtain a differential multivariate sample vector; Perform short-time Fourier transform on the time series multivariate sample vector to obtain a frequency domain multivariate sample vector.

3. The method for detecting time series change points of multi-sensor information using a fusion confidence strategy according to claim 2 is characterized in that: The time series multivariate sample vector is self-differentiated, including: for a time series multivariate sample vector of length n, the first n-1 groups of time series multivariate sample vectors v1 and the last n-1 groups of time series multivariate sample vectors v2 are taken. The result of the self-differentiation method is a differential multivariate sample vector v3 of length n-1 obtained by v2-v1.

4. The method for detecting time series change points of multi-sensor information using a fusion confidence strategy according to claim 1, characterized in that: The sequential sub-neural network model and the differential sub-neural network model adopt a bidirectional LSTM neural network; The frequency domain sub-neural network model adopts a two-layer interactive transformer network; The fusion sub-neural network model adopts a multi-layer fully connected structure.

5. The method for detecting time series change points of multi-sensor information integrated with confidence strategy according to claim 1 is characterized in that: Bayesian risk L k The calculation formula for (Θ) is: where ψ(·) is the digamma function; k represents the kth sample, y k ={y kp ,y kn } is a two-dimensional one-hot vector, indicating whether the actual category of the k-th sample is positive or negative. Positive refers to the change of the device under test, corresponding to y k ={0,1}, negative refers to the device under test does not change, corresponding to y k ={1,0};α k ={α kp ,α kn } is the Beta distribution density parameter on the predictor variable, p k ={p kp ,p kn } is the probability of being predicted as a change point or a non-change point; Be(p k |α k ) is regarded as the likelihood B(y k |p k ), Be(·) is the Beta distribution function, and B(·) is the binomial distribution function; Divergence loss KL[Be(p|α p ,α n )||Be(p|1,1)] is calculated as: in is the prediction parameter α from sample k k The Beta parameter after removing non-misleading evidence, Be(p|a,b) corresponds to the Beta distribution with parameters a and b in statistics; is the result after removing non-misleading evidence from the evidence that the kth sample in the batch is a changed sample, e k It is the neural network judgment evidence, corresponding to the intermediate output of the neural network.

6. The method for detecting time series change points of multi-sensor information integrated with confidence strategy according to claim 1, characterized in that: The self-distillation training adopts multi-model mutual KL loss: The t ,And d ,And f )=D KL "(And t ||and d )+D KL "(And d ||and f )+D KL "(And f ||and t ) Where L(e t ,e d ,e f ) is the multi-model mutual KL loss, D KL ″(a||b) is the exponential KL divergence loss, e t ,e d ,e f They refer to the features extracted from samples by the time-series sub-neural network, the difference sub-neural network, and the frequency-domain sub-neural network, respectively, and correspond to the intermediate outputs of the sub-neural networks after weighting and before summation; Where a and b refer to sub-model a and sub-model b respectively, a(x i ) and b(x i ) refer to the amount of evidence output by sub-model a and sub-model b for category i, i∈{p,n} represents the category judged as changed and the category judged as unchanged, e (·) Indicates taking the natural exponent.

7. The method for detecting time series change points of multi-sensor information integrated with confidence strategy according to claim 1, characterized in that: The self-distillation training refers to fixing the parameters of a neural network model, called the Teacher Model, and updating the parameters of other neural network models, called the Student Model, within a certain period.

8. A device for detecting time series change points of multi-sensor information integrated with a confidence strategy, characterized in that: including processor and storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images