Data extraction multi-task management method and system in railway transportation statistics

Abstract

The invention provides a data extraction multi-task management method and system in railway transportation statistics. The method comprises the following steps that: creating a thread pool which contains a plurality of threads; generating a plurality of data extraction tasks; under a situation that the created thread pool contains an idle data reading thread, utilizing the idle thread to execute adata reading process in the data extraction tasks so as to read data from a data source; under a situation that the created thread pool does not contain the idle data reading thread, adding data extraction tasks read by data to be executed into a queue to be read in sequence; under the situation that the created thread pool contains an idle data writing thread, utilizing the idle thread to execute a data writing process in the data extraction task, writing the data read from the data source into a database; and under the situation that the created thread pool does not contain the idle data writing thread, adding the data extraction tasks in a data writing process to be executed into the queue to be written in sequence. By use of the method, original data and statistics data are quickly and efficiently extracted from the professional statistical system of each grassroots unit so as to avoid server timeout or crash.

Description

technical field [0001] The invention relates to the technical field of railway transportation statistics and supervision, in particular to a data extraction multi-task management method and system in railway transportation statistics. Background technique [0002] The statistical supervision of railway wagons is the most important part of the statistical supervision of railway transportation. It plays an important role and significance in improving the production efficiency of railway transportation. In actual statistical work, it is necessary to collect basic data resources such as pre-confirmed reports, existing vehicles, and freight. is a very complex system. [0003] Based on the traditional supervision model, supervisors usually go to the scene to conduct spot checks on statistical data, and the coverage and time span of supervision work are strictly limited. With the development of railway informatization construction, the professional statistical work of railway tran...

AI Technical Summary

Problems solved by technology

At present, in the process of data collection, there are the following problems: (1) The deployment of the statistical monitoring system is scattered, and it is time-consuming and laborious to collect data: each system has a wide coverage in grassroots units, and the deployment is scattered. collect

(2) The amount of data is huge, and the pressure of data acquisition is too great: because the statistical system generates a lot of business data every day, the method of collecting the original data of the statistical system one by one cannot meet the requirements for data acquisition, and it is easy to cause the system server to crash

[0005] Therefore, in the statistical supervision of railway transportation, it is necessary to obtain a large amount of statistical data from multiple professional statistical systems. How to quickly and efficiently extract raw data and statistical data from the professional statistical systems of various grassroots units to prevent server timeouts or crashes, is an unresolved problem in the prior art

Images