Logistics report multi-source heterogeneous data cleaning method and device, equipment and storage medium

Abstract

The invention relates to the field of data processing, and discloses a logistics report multi-source heterogeneous data cleaning method, device and equipment and a storage medium, and the method is used for cleaning logistics multi-source heterogeneous report data. The method comprises the steps of obtaining multi-source heterogeneous report data, and matching the multi-source heterogeneous report data through a preset feature matching rule to obtain target data; performing exception preliminary screening and exception processing on the target data to obtain processed data; building a conflict detection model based on a CNN-LSTM hybrid neural network, and carrying out the recognition and classification of the processed data through the conflict detection model, and obtaining different types of conflict data; calling a repair algorithm corresponding to the conflict type to repair different types of conflict data to obtain repaired data; and performing full-dimensional quality verification on the repaired data based on a pre-trained data quality evaluation model, and if the verification is passed, completing data cleaning.

Images