This invention belongs to the field of
data processing, and specifically relates to a method and
system for recompressing data files based on a
large model. The
system includes a
data warehouse storage management platform; the
data warehouse storage management platform includes a
data file storage module for storing data files; a data asset ledger storage module for storing a data asset ledger containing data files, the data asset ledger including physical attribute information and business attribute information of the data files; a preliminary disposal plan generation module, which calls a
processing plan generation model, takes the data asset ledger as input, outputs a preliminary plan for the data files, and sends it to the business end; a final disposal plan generation module, which generates a final
processing plan based on the plan manually confirmed by the business end; and an
automatic data processing module, which automatically processes the data according to the final
processing plan. This invention solves the problems in existing technologies where outdated
big data clusters do not support tiered cold and hot storage, and where
business personnel misuse compression algorithms leading to storage waste.