A large capacity data and service management system

Abstract

The invention belongs to the technical field of data management service, in particular to a large-capacity data and service management system. The large-capacity data and service management device ofthe invention improves the efficiency of file data transmission and storage, Increases resource consumption for data operations such as queries, can significantly reduce the number of nodes in the data file, improve the efficiency of the whole system operation in the large-capacity data system, the burden of system file processing is decreased, the file association structure based on dependency relation simplifies the difficulty of file location and management, simplifies the location operation of dependency node in the process of file location and clustering, reduces the thread consumption offile data reading and writing, and improves the reading and writing speed.

Description

technical field [0001] The invention belongs to the technical field of data management services, in particular to a large-capacity data and service management system. Background technique [0002] With the development of information transmission and detection technology, comprehensive data processing technology is more and more widely used in daily learning, life and manufacturing processes. By setting up a variety of data acquisition devices and transmission devices, a wider range or more comprehensive data can be achieved. Information collection and processing efficiency, relatively speaking, the greater the total amount of data, the higher the accuracy and comprehensiveness of the data, but on the other hand, a large number of data sources represent a huge pressure on data transmission and processing, especially in In the application environment of production control and real-time acquisition, different from general daily monitoring and continuous transmission, these appl...

AI Technical Summary

Problems solved by technology

[0002] With the development of information transmission and detection technology, comprehensive data processing technology is more and more widely used in daily learning, life and manufacturing processes. By setting up a variety of data acquisition devices and transmission devices, a wider range or more comprehensive data can be achieved. Information collection and processing efficiency, relatively speaking, the greater the total amount of data, the higher the accuracy and comprehensiveness of the data, but on the other hand, a large number of data sources represent a huge pressure on data transmission and processing, especially in In the application environment of production control and real-time acquisition, different from general daily monitoring and continuous transmission, these application environments contain a large number of scattered small file data, such as sensor monitoring data, control signal data, etc. The individual capacity of these data is small, but the overall The number is huge, and the actual file transfer, cache, and storage system's single space / byte length are generally higher than these data, resulting in insufficient equipment efficiency in the process of processing, storage, and caching, and at the same time bringing a lot of such data concurrently. Extremely high system load burden

Images