Multi-task visual activation method of ten-million-magnitude data in relational database

Abstract

The invention discloses a multi-task visual activation method for ten-million-magnitude data in a relational database. The method comprises the steps: inputting the corresponding parameters accordingto a preset task template to generate a data import task; uploading a corresponding data source to a specified FTP path according to the selected data import task; starting a shell script program to decompose the file of the data source into sub-files, and identifying the sub-files by adopting suffixes; synchronously analyzing all the sub-files by utilizing a background Java multi-thread program,and importing the sub-files into a database in a one-to-one correspondence manner according to the suffix identifiers. According to the technical scheme, the pressure of the front end is transferred to the background, the data storage time is shortened through multi-thread operation, uploading and storage of a data source are asynchronously achieved, and the problem that a large number of levels of data cannot be automatically and visually stored due to long-time waiting of the front end is solved.

Description

technical field [0001] The invention relates to the technical field of data management, in particular to a multi-task visualization activation method for tens of millions of data in a relational database. Background technique [0002] In the IT system, after the daily batch data analysis and packaging instructions are more than 100,000, the efficiency of visually implementing the front-end interface to import the database is significantly reduced, and the waiting time is long, and the user perception is significantly reduced. The system on the market often limits the number of imported rows. The reason is that The interface and relational data are implemented in one thing, and the submission of a thing depends on the completion of the database storage. For a tens of millions of data, it needs to be imported hundreds of times or even impossible, and we often encounter large batches of data in daily systems. In the case of collectively signing a contract with network elements,...

AI Technical Summary

Problems solved by technology

[0004] Aiming at at least one of the above problems, the present invention provides a multi-task visualization activation method for tens of millions of data in a relational database. By splitting the data import task, the generation task and monitoring are performed on the front-end visual interface. The operation of tasks, start and stop tasks, the storage operation of data source files in the background, transfer the pressure of the front-end to the background and shorten the time of data storage through multi-threaded operations, realize the upload and storage of data sources asynchronously, and solve the problem of long-term front-end Waiting caused the problem that a large amount of data cannot be automatically visualized and put into storage

Images