Out-of-distribution data detection method and device, server and storage medium

Abstract

The embodiment of the invention discloses an out-of-distribution data detection method and device, a server and a storage medium. The method comprises the steps of obtaining to-be-detected data; encoding the to-be-detected data to obtain low-dimensional feature data; processing the low-dimensional feature data based on preset first memory data to obtain first low-dimensional data; processing the low-dimensional feature data based on preset second memory data to obtain second low-dimensional data; generating data to be decoded based on the first low-dimensional data and the second low-dimensional data; decoding the to-be-decoded data to obtain reconstructed data; determining a reconstruction error of the to-be-detected data and the reconstruction data, and if the reconstruction error is greater than a preset threshold, determining that the to-be-detected data is out-of-distribution data. According to the embodiment of the invention, the reconstruction capability of the self-encoder on the OOD data is limited, and the difference between the reconstruction errors of the ID data and the OOD data is improved, so the accuracy of OOD data identification is improved.

Description

technical field [0001] The embodiments of the present invention relate to the technical field of machine learning, and in particular to a method, device, server and storage medium for detecting out-of-distribution data. Background technique [0002] In the field of machine learning, the data used to train the model is usually referred to as In-distribution (ID) data, while Out-of-distribution (OOD) data refers to data that is inconsistent with the training ID data distribution. . In the practical application of neural network models, there are sometimes OOD data in the input data, which will lead to inaccurate model predictions and limit the application of neural network modules. Therefore, OOD data detection for the input data of the machine learning model is an important means to improve the prediction accuracy of the model. [0003] At present, the detection of OOD data mainly adopts the OOD detection algorithm based on autoencoder. The autoencoder uses ID data for tra...

AI Technical Summary

Problems solved by technology

However, due to the strong generalization ability of the autoencoder, even the OOD data outside the distribution of the training data may be well reconstructed, which makes the autoencoder unable to detect the OOD data well.

Images