Data updating method, distributing node and mass storage cluster system

Abstract

The invention discloses a data updating method, a distributing node and a mass storage cluster system. The distributing node comprises an order control module and a distributing module, wherein the order control module is used for receiving an updated order sent by a thread receiving and managing device; and the distributing module is used for reading the updated order from the order control module via a plurality of reading threads, and respectively distributes data to the corresponding data nodes according to the updated order. The data updating method, the distributing node and the mass storage cluster system are characterized in that only one updating thread is required to receive the updated order, thereby avoiding the lock problem that multiple threads are simultaneously updated in the prior art, and improving the integral processing efficiency of the system. Meanwhile, a plurality of reading threads read the updated orders, data are distributed to a plurality of updated orders simultaneously, and the data distributing efficiency is improved.

Description

Technical field [0001] The invention relates to a data storage technology, in particular to a data update method, a distribution node and a mass storage cluster system. Background technique [0002] It is difficult to realize real-time query and synchronous update of massive data in various existing real-time systems. At present, common strategies are to increase the data carrying capacity by improving the performance of single-machine hardware, or by dividing the storage unit by data according to different usage methods. . [0003] The existing technology has the following defects: [0004] 1. Due to the simultaneous update of multiple threads, the system needs to select an update thread and lock other update threads. There are random I / O operations when processing updates, which makes it difficult to improve overall efficiency. Most of the efficiency loss lies in I / O lock competition and Update thread scheduling. [0005] 2. The same thread is used for query and update, resulting ...

AI Technical Summary

Problems solved by technology

[0004] 1. Due to the simultaneous update of multiple threads, the system needs to select one update thread and lock other update threads. There are random I / O operations during update processing, which makes it difficult to improve the overall efficiency. Most of the efficiency loss is due to I / O lock competition and update thread scheduling

[0005] 2. Query and update use the same thread, resulting in ambiguous permissions and uncertain data flow. If there is a problem with query or update, it will affect the entire system

[0006] 3. In order to ensure data security, many systems adopt multi-copy mode, which means that one copy is stored in 2 or 3 data nodes, thereby improving data security. This method seems safe but actually has data security risks. When the system When there is a bug in the code, multiple data copies may be damaged or lost; and when the data is synchronized, if A notifies B and B notifies C of the update method one by one, the update will not be timely, and each node will be updated The key node of synchronization, the data between each node is prone to inconsistency

[0007] 4. With the growth of data scale, excessive reliance on a certain data center node leads to excessive pressure on the central node, performance degradation, and difficulty in horizontal expansion of the system

[0008] 5. Many business systems require 7*24 hours of uninterrupted service every week. When the amount of data is small, it can be guaranteed by several redundant servers. However, in the case of massive data, it is necessary to consider that each distribution node in the entire system is offline Moreover, when the machine scale reaches a certain level, such as a thousand machines, there may be machine damage every week or even every day, which will have a great impact on the normal operation of the system

Images