File data writing method and device and computer readable storage medium

Abstract

The invention relates to a big data technology, and discloses a file data writing method, which comprises the following steps of: storing a plurality of acquired files and generating a file task list;calling files in the file task list to gather the files into a temporary file set, and classifying the files in the temporary file set to obtain a classification task table; selecting the files of the same classification in the classification task table and putting the files into a preset task queue; when vacancies appear in the task queue, selecting the files of the same classification to be putinto the task queue until all the files of the same classification in the classification task list are put into the task queue, selecting other files of the same classification to be continuously putinto the task queue, and writting the files in the task queue into preset electronic equipment. The invention further provides a file data writing device, electronic equipment and a computer readablestorage medium. According to the file data writing method and the file data writing device, the problems of low writing efficiency and high cost of ten-million-level file data can be solved.

Description

technical field [0001] The present invention relates to the field of big data technology, and in particular to a file data writing method, device, electronic equipment, and computer-readable storage medium. Background technique [0002] With the rise of big data, people will generate huge amounts of data in their daily network activities, such as browsing the web, uploading data, downloading data, etc. These generated data will form tens of millions of file data, so for tens of millions There are more and more file data processing requirements at the same level. [0003] However, due to the huge amount of data contained in tens of millions of file data, when it is written, it takes up too much computing memory and the writing efficiency is low. Therefore, how to achieve tens of millions of files with low cost and high efficiency The demand for writing file data is increasing. Contents of the invention [0004] The present invention provides a file data writing method, de...

AI Technical Summary

Problems solved by technology

[0004] The present invention provides a file data writing method, device, electronic equipment and computer-readable storage medium, the main purpose of which is to solve the problem of low efficiency and high cost of writing tens of millions of file data

Images