Task scheduling system based on thread pool

Abstract

The invention provides a task scheduling system based on a thread pool, and the system comprises a task storage module which comprises a redis fragment cluster unit and a mysql data unit; wherein the redis fragment cluster unit stores a plurality of numerical pairs < key, value >, wherein the key is a main key, and the value is a trigger condition; the mysql data unit stores a primary key and a corresponding task class; a task scanning module that scans and obtains the task class from the task storage module, and transmits the task class to a task execution module; and the task execution module that executes a preset task according to the task class. A plurality of value pairs < key and value > is stored by redis, data needing to be scanned in real time are only stored in a memory, the memory space is fully utilized and saved, and reading speed is increased; the data is stored in redis and mysql data units, so that task data stored in the system is maximized; the triggering condition and the task type data are associated through the primary key, and the working efficiency of the system is improved.

Description

technical field [0001] The invention relates to the technical field of thread pools, in particular to a thread pool-based task scheduling system. Background technique [0002] At present, most open source task scheduling frameworks, such as the professional framework quartz and the system task module of activiti, have the function of task scheduling. The above frameworks are based on the thread pool to scan and execute tasks; when the amount of task data is small, save the task data in the memory to scan quickly; when the amount of task data increases, in order to avoid memory overflow, most of the tasks The data is saved to the database, so it is less efficient to scan the data. Since all task data needs to be scanned, a full table scan is required, the scanning efficiency is low, and there are also high requirements for the performance of the machine. [0003] With the gradual rise and widespread application of nosql databases such as redis, data storage is no longer lim...

AI Technical Summary

Problems solved by technology

The amount of data stored by a single machine is related to the memory size of the machine itself, that is to say, the amount of data stored by redis is limited, and redis has the problem of double-write consistency between memory and database

Images