Out-of-distribution detection method based on spatio-temporal feature inductive conformal analysis

Through the combination of spatiotemporal feature induction and dual input variational autoencoder, combined with the inductive conformal anomaly detection of martingale, the accuracy and false alarm rate of extradistribution detection in time series data are solved, achieving high accuracy and low information Extra-distribution detection of loss.

CN119066376APending Publication Date: 2024-12-03DALIAN UNIV OF TECH
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410906467.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-08
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

In the out-of-distribution detection of time series data, existing methods are difficult to provide high accuracy and bounded false alarm rates, and there is also the problem of information loss.

Method used

The conformal analysis method based on spatiotemporal feature induction is adopted to extract continuous spatiotemporal data features through optical flow constraints, and a dual-input variational automatic encoder is used to form horizontal and vertical hidden secret spaces. Combined with the inductive conformal anomaly detection method of martingale, a hybrid is constructed Martingale value is used to correct the data outside the distribution.

Benefits of technology

It significantly improves the accuracy and robustness of off-distribution detection, reduces information loss, and provides bounded false alarm rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066376A_ABST
    Figure CN119066376A_ABST
Patent Text Reader

Abstract

The invention relates to an out-of-distribution detection method based on spatio-temporal feature inductive conformal analysis, and belongs to the technical field of out-of-distribution detection. The method comprises the following steps of: 1, extracting continuous spatio-temporal data characteristics in out-of-distribution detection of time sequence data; 2, training by using a double-input variational automatic encoder according to the continuous spatio-temporal data features obtained in the step 1; 3, according to the divergence difference between the sample obtained in the step 2 and the training, adopting a yoke value quantization method in a yoke inductive shape-preserving anomaly detection method to construct a mixed yoke; thus, out-of-distribution detection with bounded false alarm rate but reduced information loss is provided. According to the method, the detection capability of out-of-distribution detection data is improved, the information loss is reduced, and the robustness of out-of-distribution detection is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to an out-of-distribution detection method based on spatiotemporal feature induction conformal analysis, belonging to the technical field of out-of-distribution detection. Background Art

[0002] In autonomous systems, data-driven learning-enabled components achieve effective perception, planning, and control functions by learning "experience knowledge" from large amounts of data. However, in real autonomous systems, the out-of-distribution data generated by the significant distribution difference between runtime data and training data due to the uncertainty and variability in the distribution change environment can cause data-driven learning components to make incorrect predictions.

[0003] Currently, existing out-of-distribution data detection methods are generally divided into: independent data point data out-of-distribution detection based on feature recognition of out-of-distribution instances based on single data points and out-of-distribution detection of time series data for time window training out-of-distribution.

[0004] In the out-of-distribution detection of time series data, Feng et al. proposed an out-of-distribution detection method based on variational autoencoder. This method uses the spatiotemporal data features in the known video to detect out-of-distribution data by simply summing the KL (Kullback-Leibler) divergence in the latent subspace of the autoencoder, without making full use of other methods to improve the accuracy of the detection results.

[0005] On the false alarm of out-of-distribution detection of time series data, Kaur et al. proposed a conformal out-of-distribution detection method called CODiT. This method uses the equal variance deviation of in-distribution data in time as the failure metric in the conformal anomaly detection framework to combine multiple conformal detection independent prediction results to provide a bounded false alarm rate, but the out-of-distribution detection performance is impaired due to the sensitivity of its distribution assumptions and potential information loss.

[0006] Therefore, in the out-of-distribution detection of time series data, how to provide high-accuracy out-of-distribution detection for time series data and provide a bounded false alarm rate and reduce information loss has become an urgent problem to be solved. Summary of the invention

[0007] The purpose of the present invention is to solve the technical problems of providing high-accuracy out-of-distribution detection in time series data and providing a method with a bounded false alarm rate and reduced information loss, and proposes an out-of-distribution detection method based on spatiotemporal feature induction conformal analysis.

[0008] The objective of the present invention is achieved through the following technical solutions:

[0009] The present invention provides an out-of-distribution detection method based on spatiotemporal feature induction conformal analysis, comprising the following steps:

[0010] Step 1: Extraction of continuous spatiotemporal data features in out-of-distribution detection of time series data.

[0011] Step 1.1: Use the optical flow constraint method to obtain each image frame X during the short interval t to t+Δt of the video t The pixel intensity at position (x, y) and time t is I(x, y, t), as shown in equation (1).

[0012] I(x+Δx,y+Δy,t+Δt)=I(x,y,t) (1)

[0013] where Δx and Δy are the changes in position, and Δt is the change in time. According to the brightness consistency and velocity smoothness assumptions, although the object may change position, its reflectivity and illumination will remain unchanged.

[0014] Step 1.2: Construct the optical flow constraint equation according to the pixel intensity obtained in step 1.1, as shown in equation (2).

[0015]

[0016] in They are the partial derivatives of the pixel intensity function in the corresponding dimensions of x, y, and t, respectively, and hot is a high-order term that is ignored.

[0017] Step 1.3: The optical flow constraint equation in step 1.2 is constrained by the unit time displacement between any two consecutive adjacent frames in the time series, i.e., Δt=1, as shown in formula (3).

[0018]

[0019] Step 1.4: Extract the continuous spatiotemporal data features, i.e., the optical flow vector v, between any two consecutive adjacent frames in the time series, as shown in formula (4).

[0020] ▽I·v+I t =0 (4)

[0021] in is the spatial gradient, is the temporal gradient, and v = (Δx, Δy) is the optical flow vector.

[0022] Step 2: The continuous spatiotemporal data features obtained in step 1 are trained using a dual-input variational autoencoder.

[0023] Step 2.1: Use a dual-input variational autoencoder to train from the horizontal and vertical directions of the spatiotemporal data features to form the horizontal and vertical latent subspaces of the training.

[0024] Step 2.2: Construct the objective function of the dual-input variational autoencoder network as shown in Equation (5).

[0025]

[0026] Where p(Z v ) and p(Z h ) is the prior distribution, q(Z v |Δy) is the variational distribution of the dual-input variational autoencoder, The reconstruction capability of the decoder is the maximum likelihood between the input and reconstruction of the variational autoencoder. KL (q(Z h |Δx)||p(Z h ) and D KL (q(Z v |Δy)||p(Z v ) represents the ability of the model to learn meaningful features, which is the difference in Kullback-Leibler divergence between the approximate posterior and prior in the horizontal and vertical latent subspaces, respectively.

[0027] Step 2.3: The horizontal and vertical latent subspaces described in step 2.1 are used as inputs of the objective function of the dual-input variational autoencoder described in step 2.2, and training data with horizontal and vertical latent subspaces is obtained through dual-input variational autoencoder training.

[0028] Step 2.4: Use the Kullback-Leibler divergence to obtain the non-compliant scoring function A(z) in the inductive conformal anomaly detection by summing the divergence differences of the horizontal and vertical latent subspaces of the sample data distribution of the dual-input variational autoencoder relative to the training data distribution obtained in step 2.3. k ), as shown in formula (6), is used to measure the inconsistency between the sample data distribution and the training data distribution.

[0029]

[0030] Where n is the dimension of the latent subspace of the two-input variational autoencoder, and They are sample z k Approximate posterior distribution representation in horizontal and vertical directions in variational autoencoder, dk h and dk v They are sample z k The true prior distribution in the horizontal and vertical directions.

[0031] Step 3: Based on the divergence difference between the sample obtained in step 2 and the training, the martingale value quantization in the inductive conformal anomaly detection method of martingale is used to construct a hybrid martingale. Thus, out-of-distribution detection with bounded false alarm rate and reduced information loss is provided.

[0032] Step 3.1: Use the P-value (z k ) is shown in formula (7), which is used to quantify the confidence of the out-of-distribution detection results. Thus, the out-of-distribution data detection capability is improved.

[0033]

[0034] in is the calibration set T Cal The latent space representation of the variational autoencoder of the sequence fragment in is the sampled sample, and c is the calibration set Τ Cal The total number of sequence fragments.

[0035] Step 3.2: Construct the mixed martingale value set M s The out-of-distribution detection data is corrected using the method shown in equation (8), thereby providing out-of-distribution detection with bounded false alarm rate and reduced information loss.

[0036]

[0037] where ε∈(0,1), s is the number of p-value sets, M s Used to determine out-of-distribution detection data.

[0038] The present invention utilizes the construction of hybrid martingales in the inductive conformal anomaly detection method of martingales in step 2.4 and step 3 to respectively solve the problem of providing high-accuracy out-of-distribution detection in time series data and providing a bounded false alarm rate and reducing information loss.

[0039] Beneficial Effects

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] The present invention discloses an out-of-distribution detection method based on spatiotemporal feature inductive conformal analysis, which utilizes the extracted spatiotemporal data features to obtain the divergence difference of the horizontal and vertical latent subspaces of the sample data distribution relative to the training data distribution through a dual-input variational autoencoder, and corrects the out-of-distribution detection data through a mixed martingale in the martingale inductive conformal anomaly detection method, thereby greatly improving the detection capability of the out-of-distribution detection data and reducing information loss, thereby improving the robustness of the out-of-distribution detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the process of the present invention;

[0043] Figure 2 It is a schematic diagram of the experimental results of the drift data set of the present invention;

[0044] Figure 3 It is a schematic diagram of the experimental results of the Carla data set of the present invention. DETAILED DESCRIPTION

[0045] In order to better illustrate the purpose and advantages of the present invention, the invention is further described below in conjunction with the accompanying drawings and examples. It should be noted that the implementation of the present invention is not limited to the following embodiments, and any form of modification or change made to the present invention will fall within the protection scope of the present invention.

[0046] Example

[0047] like Figure 1 As shown, the out-of-distribution detection method based on spatiotemporal feature induction conformal analysis of the present invention is specifically implemented in the following steps:

[0048] The in-distribution dataset Τ for training is divided into two disjoint subsets: training set Τ tr and calibration set Τ Cal Training set Τ tr Used to train the variational autoencoder, the calibration set Τ Cal It is used as a reference data set for calculating the p-value. For a test video sequence data, it is defined as a time sliding window χ i And evenly divided into s sequence segments, each sequence segment has 7 continuous image frames;

[0049] Step 1: Extraction of continuous spatiotemporal data features in out-of-distribution detection of time series data.

[0050] Step 1.1: Use the optical flow constraint method to obtain each image frame X during the short interval t to t+Δt of the video t The pixel intensity at position (x, y) and time t is I(x, y, t), as shown in equation (9).

[0051] I(x+Δx,y+Δy,t+Δt)=I(x,y,t) (9)

[0052] where Δx and Δy are the changes in position, and Δt is the change in time. According to the brightness consistency and velocity smoothness assumptions, although the object may change position, its reflectivity and illumination will remain unchanged.

[0053] Step 1.2: Construct the optical flow constraint equation according to the pixel intensity obtained in step 1.1, as shown in equation (10).

[0054]

[0055] in They are the partial derivatives of the pixel intensity function in the corresponding dimensions of x, y, and t, respectively, and hot is a high-order term that is ignored.

[0056] Step 1.3: The optical flow constraint equation in step 1.2 is constrained by the unit time displacement between any two consecutive adjacent frames in the time series, i.e., Δt=1, as shown in equation (11).

[0057]

[0058] In the embodiment, when the time displacement Δt=1, the optical flow constraint equation shown in formula (11) is used to constrain each sequence segment, that is, 7 consecutive image frames.

[0059] Step 1.4: Extract the continuous spatiotemporal data features, i.e., the optical flow vector v, between any two consecutive adjacent frames in the time series, as shown in formula (12).

[0060] ▽I·v+I t =0 (12)

[0061] in is the spatial gradient, is the temporal gradient, and v = (Δx, Δy) is the optical flow vector.

[0062] Step 2: The continuous spatiotemporal data features obtained in step 1 are trained using a dual-input variational autoencoder.

[0063] Step 2.1: Use a dual-input variational autoencoder to train from the horizontal and vertical directions of the spatiotemporal data features to form the horizontal and vertical latent subspaces of the training.

[0064] Step 2.2: Construct the objective function of the dual-input variational autoencoder network as shown in Equation (13).

[0065]

[0066] Where p(Z v ) and p(Z h ) is the prior distribution, q(Z v |Δy) is the variational distribution of the dual-input variational autoencoder, The reconstruction capability of the decoder is the maximum likelihood between the input and reconstruction of the variational autoencoder. KL (q(Z h |Δx)||p(Z h ) and D KL (q(Z v |Δy)||p(Z v) represents the ability of the model to learn meaningful features, which is the difference in Kullback-Leibler divergence between the approximate posterior and prior in the horizontal and vertical latent subspaces, respectively.

[0067] Step 2.3: The horizontal and vertical latent subspaces described in step 2.1 are used as inputs of the objective function of the dual-input variational autoencoder described in step 2.2, and training data with horizontal and vertical latent subspaces is obtained through dual-input variational autoencoder training.

[0068] In the embodiment, after encoding by a dual-input variational autoencoder, a data distribution in the horizontal and vertical latent subspaces is obtained. S samples {z1…z k …z s}.

[0069] Step 2.4: Use the Kullback-Leibler divergence to obtain the non-compliant scoring function A(z) in the inductive conformal anomaly detection by summing the divergence differences of the horizontal and vertical latent subspaces of the sample data distribution of the dual-input variational autoencoder relative to the training data distribution obtained in step 2.3. k ), as shown in formula (14), is used to measure the inconsistency between the sample data distribution and the training data distribution.

[0070]

[0071] Where n is the dimension of the latent subspace of the two-input variational autoencoder, and They are sample z k Approximate posterior distribution representation in horizontal and vertical directions in variational autoencoder, dk h and dk v are the true prior distributions of sample zk in the horizontal and vertical directions, respectively.

[0072] In the embodiment, for a sliding time window x i , divide it into s sequence segments on average For each segment, the latent subspace representation in the horizontal and vertical directions can be obtained by using the trained dual-input variational autoencoder. i By sampling the latent space representation of the variational autoencoder of each sequence segment, s samples {z1…z k …z s}.

[0073] Step 3: Based on the divergence difference between the sample obtained in step 2 and the training, the martingale value quantization in the inductive conformal anomaly detection method of martingale is used to construct a hybrid martingale. Thus, out-of-distribution detection with bounded false alarm rate but reduced information loss is provided.

[0074] Step 3.1: Use the P-value (z k ) is shown in formula (15), which is used to quantify the confidence of the out-of-distribution detection results. Thus, the out-of-distribution data detection capability is improved.

[0075]

[0076] in is the calibration set T Cal The latent space representation of the variational autoencoder of the sequence fragment in is the sampled sample, and c is the calibration set Τ Cal The total number of sequence fragments.

[0077] In the embodiment, when This means that when the test sample is smaller than the calibration set, the p-value (z k ) is incremented by 1. When the test sample is not smaller than the calibration set, the p-value (z k ) is 0, the correct result of confidence cannot be obtained. Therefore, in formula (15) Add 1 to ensure the correct result. The applicable range of p value is (0, 0.3)

[0078] Step 3.2: Construct the mixed martingale value set M s The out-of-distribution detection data is corrected using the method shown in equation (16), thereby providing out-of-distribution detection with bounded false alarm rate but reduced information loss.

[0079]

[0080] where ε∈(0,1), s is the number of p-value sets, M s Used to determine out-of-distribution detection data.

[0081] In the embodiment, for this section of test video data x i , we can use the obtained s p values ​​to construct a mixed martingale value set s and use the formula (16) to determine the out-of-distribution detection data. k ∈{z1…z k …z s}, if the martingale value p-value(z k ) is less than the predefined abnormal threshold ε∈(0,1), then sample z k ∈{z1…z k …z s} can be summarized as a conformal anomaly, and then related to zk The relevant time series window is the out-of-distribution data window. i , sampled from the latent space representation of the variational autoencoder of s sequence segments, and obtain s samples {z1…z k …z s}, and then get s p values ​​{p1…p k …p s} is used to detect out-of-distribution window data. k …p s}When the p-value is small by using the mixed martingale value for out-of-distribution data detection, M s The larger the value, the lower the information loss.

[0082] To verify the effectiveness of the present invention, the present invention is experimentally compared with the existing Feng and CODiT in the time series distribution data detection method to verify the effectiveness and robustness of the present invention. The experimental samples use the real environment car drift data set and the simulated driving data set under sunny weather and snowy weather conditions generated by the open source simulator Carla. Under the condition of 95% true positive rate (TPR), F1 score, area under the receiver operating characteristic curve (AUROC), precision, and false detection rate (FDR) are used as detection indicators.

[0083] like Figure 2 and Figure 3 As shown in the figure, the present invention has better detection robustness than Feng and CODiT in terms of F1 score detection index. For the real driving drift data set, ST-OOD enhances CODiT, with an accuracy improvement of 15.58% and a FDR reduction of 52.17%. The results show that the out-of-distribution detection method based on spatiotemporal feature induction conformal analysis proposed in the present invention can solve the problem of providing high accuracy out-of-distribution detection in time series data and providing a bounded false alarm rate and reducing information loss.

Claims

1. An out-of-distribution detection method based on spatiotemporal feature induction conformal analysis, characterized in that: The following steps are included: Step 1: Extraction of continuous spatiotemporal data features in out-of-distribution detection of time series data; Step 1.1: Use the optical flow constraint method to obtain each image frame X during the short interval t to t+Δt of the video t The pixel intensity at position (x, y) and time t is I(x, y, t), as shown in equation (1); I(x+Δx,y+Δy,t+Δt)=I(x,y,t) (1) where Δx and Δy are the changes in position, and Δt is the change in time; according to the brightness consistency and velocity smoothness assumptions, although the object may change position, its reflectivity and illumination will remain unchanged; Step 1.2: Construct the optical flow constraint equation according to the pixel intensity obtained in step 1.1, as shown in formula (2); in are the partial derivatives of the pixel intensity function in the corresponding dimensions of x, y, and t, respectively, and hot is a high-order term that is ignored; Step 1.3: Constrain the optical flow constraint equation in step 1.2 by the unit time displacement between any two consecutive adjacent frames in the time series, i.e., Δt=1, as shown in formula (3); Step 1.4: Extract the continuous spatiotemporal data features between any two consecutive adjacent frames in the time series, namely, the optical flow vector v, as shown in formula (4); in is the spatial gradient, is the temporal gradient, v = (Δx, Δy) is the optical flow vector; Step 2: Train the continuous spatiotemporal data features obtained in step 1 using a dual-input variational autoencoder; Step 2.1: Use a dual-input variational autoencoder to train from the horizontal and vertical directions of the spatiotemporal data features to form the horizontal and vertical latent subspaces of the training; Step 2.2: Construct the objective function of the dual-input variational autoencoder network as shown in formula (5); Where p(Z v ) and p(Z h ) is the prior distribution, q(Z v |Δy) is the variational distribution of the dual-input variational autoencoder, Denotes the reconstruction capability of the decoder as the maximum likelihood between the input and reconstruction of the variational autoencoder; -D KL (q(Z h |Δx)||p(Z h ) and D KL (q(Z v |Δy)||p(Z v ) represents the ability of the model to learn meaningful features, which is the difference in Kullback-Leibler divergence between the approximate posterior and prior in the horizontal and vertical latent subspaces respectively; Step 2.3: The horizontal and vertical latent subspaces described in step 2.1 are used as inputs of the objective function of the dual-input variational autoencoder described in step 2.2, and training data having horizontal and vertical latent subspaces is obtained through dual-input variational autoencoder training; Step 2.4: Use the Kullback-Leibler divergence to obtain the non-compliant scoring function A(z) in the inductive conformal anomaly detection by summing the divergence differences of the horizontal and vertical latent subspaces of the sample data distribution of the dual-input variational autoencoder relative to the training data distribution obtained in step 2.

3. k ), as shown in formula (6), is used to measure the inconsistency between the sample data distribution and the training data distribution; Where n is the dimension of the latent subspace of the two-input variational autoencoder, and They are sample z k Approximate posterior distribution representation in horizontal and vertical directions in variational autoencoder, dk h and dk v They are sample z k The true prior distribution in the horizontal and vertical directions; Step 3: According to the divergence difference between the sample obtained in step 2 and the training, the martingale value quantization in the martingale inductive conformal anomaly detection method is used to construct a hybrid martingale; Thus, out-of-distribution detection with bounded false alarm rate but reduced information loss is provided; Step 3.1: Use the P-value (z k ) is shown in formula (7), which is used to quantify the confidence of the detection results outside the calibration distribution; thereby, improving the detection capability of out-of-distribution data; in is the calibration set T Cal The latent space representation of the variational autoencoder of the sequence fragment in is the sampled sample, and c is the calibration set Τ Cal The total number of sequence fragments; Step 3.2: Construct the mixed martingale value set M s The out-of-distribution detection data is corrected by using the method shown in formula (8); thereby providing out-of-distribution detection with bounded false alarm rate but reduced information loss; where ε∈(0,1), s is the number of p-value sets, M s Used to determine out-of-distribution detection data.

2. The out-of-distribution detection method based on spatiotemporal feature induction conformal analysis as claimed in claim 1, characterized in that: The applicable range of p-value is (0, 0.3).

Citation Information

Cited By

  • Real-time drift detection in time-series data for automated AI model re-training

    US12646003B1