The present invention relates to the field of
data processing technology, and discloses a
data file archiving method and
system. The method comprises the following steps: establishing a
streaming data processing platform and configuring archiving information; scanning the data files to be archived on the
streaming data processing platform and mapping them into corresponding data files, check files and ddl files, and adding the
data file records to be archived in the
database table; reading the
database table in real time, and when a new
data file record to be archived appears, reading the new data file to be archived and recording the corresponding archiving information; comparing the file name in the archiving information with the file name of the data file corresponding to the new data
file record to be archived, if they are the same, archiving the data file corresponding to the new data
file record to be archived and storing it in a storage medium, otherwise reporting an error. The present invention can realize the
automation of the entire process of archiving historical data, avoiding the problems of long archiving time and
data delay caused by human operation interference and large amount of data accumulation.