Decentralized multi-node same-data-source data distribution method and system

Abstract

The invention relates to the field of distributed architecture, and discloses a decentralized multi-node same-data-source data distribution method and system, and the method comprises the steps: enabling a plurality of task nodes to compete for initialization authority, and enabling one task node which competes for obtaining the initialization authority to serve as an initialization node; after obtaining the initialization permission, the initialization node obtains a data source from the cloud server and stores the data source into the memory database; carrying out initialization processing on the data source to generate a data queue; and after the initialization task is completed, the initialization node also serves as a task node, and data units are extracted from the data queue according to a specified sequence to be processed. And node decentration is realized through multi-node competition initialization authority. The maintenance cost of the central node is reduced, and the response speed and the working efficiency of the system are improved.

Description

Technical Field

[0001] The present application relates to the field of distributed architecture, and more specifically, to a decentralized multi-node same-data source data distribution method and system. Background Art

[0002] Centralized nodes play a role of centralized control and coordination in a distributed architecture. They are responsible for managing the behavior of other nodes, simplifying system management, improving decision-making efficiency, and ensuring data consistency. However, this centralization also brings a series of disadvantages, including the risk of single point failure, which may cause the entire system to crash when the central node fails; performance bottlenecks, because all requests must pass through the central node, limiting the scalability of the system; increased communication overhead, especially when there are many nodes; concentrated security risks, and the central node becomes the main target of attack; and the possible increase in the risk of data privacy leakage. Summary of the invention

[0003] In order to solve the above technical problems, the present application provides a decentralized multi-node same-data source data distribution method and system, which realizes node decentralization through multi-node competition for initialization authority. It reduces the maintenance cost of the central node and improves the response speed and work efficiency of the system. Specifically, the technical solution of the present application is as follows:

[0004] In a first aspect, the present application discloses a decentralized multi-node same-data source data distribution method, comprising the following steps:

[0005] A plurality of task nodes compete for initialization authority, and a task node that obtains the initialization authority is used as an initialization node;

[0006] After obtaining the initialization authority, the initialization node obtains the data source from the cloud server and stores the data source in the memory database; and performs initialization processing on the data source to generate a data queue;

[0007] After completing the initialization task, the initialization node also serves as the task node, extracting data units from the data queue for processing in a specified order.

[0008] In some implementations, the initialization process of the data source to generate a data queue comprises the following steps:

[0009] After receiving the data source, the memory database traverses the data source and organizes all the data therein into the data queue;

[0010] The data queue is divided according to a preset data amount to obtain a plurality of data units; the initialization node increases a number for each of the data units and defines an initialization flag.

[0011] In some implementations, extracting data units from the data queue for processing in a specified order comprises the following steps:

[0012] Each of the task nodes extracts one data unit at a time for processing;

[0013] After the data unit is extracted, the memory database deletes the data unit.

[0014] In some embodiments, the decentralized multi-node same-data source data distribution method further includes: after the current data unit is processed, the task node re-extracts the next data unit until the data value of the data queue is zero.

[0015] In some embodiments, the decentralized multi-node same-data source data distribution method further includes: after data processing is completed, the node that last processes the data deletes the initialization flag.

[0016] In some implementations, the plurality of task nodes compete for initialization authority, and one of the task nodes that competes for the initialization authority serves as the initialization node; the steps include:

[0017] Before requesting permission, the task node verifies the node status to ensure that it meets the conditions for participating in the initialization permission competition;

[0018] When competition starts, the task node sends a request to the server to obtain a distributed lock according to the competition mechanism to ensure exclusive access to the data source;

[0019] If any task node is successfully acquired, the competition result is returned;

[0020] If the task node does not receive a response within the specified time, it times out and retries the request to obtain the distributed lock.

[0021] In some implementations, the initialization node has the same configuration as the task node.

[0022] In the second aspect, the present application also discloses a decentralized multi-node same-data source data distribution system, including: multiple task nodes, a cloud server, and an in-memory database; the system is used to execute a decentralized multi-node same-data source data distribution method described in any of the above-mentioned embodiments.

[0023] In some implementations, a plurality of the task nodes are used to compete for initialization authority, and a task node that competes to obtain the initialization authority serves as an initialization node;

[0024] The initialization node is used to obtain the data source from the cloud server after obtaining the initialization authority, store the data source in the memory database; and perform initialization processing on the data source to generate a data queue;

[0025] After completing the initialization task, the initialization node also serves as the task node, extracting data units from the data queue for processing in a specified order.

[0026] In some implementations, after receiving the data source, the memory database is further used to traverse the data source and organize all the data therein into the data queue;

[0027] The memory database is further used to divide the data queue according to a preset data volume to obtain a plurality of data units;

[0028] The initialization node is further used to add a number to each of the data units and define an initialization flag.

[0029] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0030] 1. This application realizes node decentralization through multi-node competition for initialization authority. It is easy to implement. When implementing, you only need to modify the configuration of multiple task nodes so that they can complete the initialization task. The R&D cost is relatively low compared to centralization, and there is less server-side development. There are few changes based on the existing distributed architecture. There is no additional equipment cost.

[0031] 2. Decentralized nodes reduce the maintenance cost of central nodes and provide higher fault tolerance. Because the system does not rely on a single node, the failure of any node will not cause the entire system to stop. New nodes can be freely added without the direct participation of central nodes, which enhances the scalability of the system. Data is no longer stored in a centralized manner, which can better protect data privacy and improve anti-attack capabilities.

[0032] 3. Decentralized nodes improve the system's response speed and work efficiency. After completing the initialization task, the initialization node still performs data processing tasks like other task nodes. Task nodes can extract and process requests from the queue without waiting for instructions from the central node. Processing can be done faster and more efficiently. After extracting data, the node deletes the data from the queue to ensure accurate and non-duplicate data retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The preferred implementation scheme will be described below in a clear and understandable manner with reference to the accompanying drawings to further illustrate the above-mentioned characteristics, technical features, advantages and implementation methods of the present application.

[0034] Figure 1 A flowchart of the steps of an embodiment of a decentralized multi-node same-data source data distribution method of the present application;

[0035] Figure 2 A schematic diagram of a process of multiple nodes competing for initialization authority in an embodiment of the present application;

[0036] Figure 3 This is a schematic diagram of the process of initializing data in an embodiment of the present application;

[0037] Figure 4 This is a schematic diagram of the process of each node taking data for processing in an embodiment of the present application;

[0038] Figure 5 This is a flowchart of deleting the initialization flag in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0040] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections.

[0041] In order to simplify the drawings, only the parts related to the invention are schematically shown in each figure, and they do not represent the actual structure of the product. In addition, in order to simplify the drawings and facilitate understanding, in some figures, only one of the parts with the same structure or function is schematically drawn or marked. In this article, "one" not only means "only one", but also means "more than one".

[0042] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0043] In this document, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0044] In a specific implementation, the terminal device described in the embodiments of the present application includes, but is not limited to, other portable devices such as mobile phones, laptop computers, tutoring machines, or tablet computers with touch-sensitive surfaces (e.g., touch screen displays and / or touch pads). It should also be understood that in some embodiments, the terminal device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch screen displays and / or touch pads).

[0045] In addition, in the description of the present application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0046] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the specific implementation methods of the present application will be described below with reference to the accompanying drawings. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings and other implementation methods can be obtained based on these drawings without creative work.

[0047] A large-scale concurrent system is a computer system that can handle a large number of user requests and transactions at the same time. Such a system usually needs to support high concurrency, that is, it can handle a large number of concurrent operations at the same time without causing significant performance degradation. Large-scale concurrent systems usually adopt a distributed architecture to spread data and services across multiple nodes to improve the system's processing power and fault tolerance.

[0048] Nodes are the basic units that make up the entire system. They work together to provide services, process tasks, and store data. Nodes help the system handle high concurrency by sharing the request load, improving the system's throughput and response speed.

[0049] For example, when processing a data distribution task, a merchant needs to distribute a certain product coupon to three million users within one hour. Assume that the merchant server can only issue 300 gift coupons per second. The number of coupons issued in one hour using one task node cannot meet the target demand, so multi-node concurrent processing is required.

[0050] Generally, the existing distributed architecture uses a central node as the control center of the system, which is responsible for coordinating and managing the behavior of other nodes to ensure the overall consistency and stability of the system. The centralized node can be used as a load balancer to dynamically assign tasks to other nodes based on the system load. At the same time, the centralized node can be used as a center for data synchronization and consistency assurance to ensure that the data status of all nodes is consistent. All decisions are made by the central node, and the management and maintenance of the system can be more centralized and simplified.

[0051] However, since all requests and data flows need to pass through the central node, this may cause the central node to be overloaded and become a performance bottleneck. If the central node fails, the entire system may stop functioning or performance may drop significantly. The central node needs to communicate with all other nodes to exchange information and data, which may increase communication delays, especially when network bandwidth is limited. Other nodes need to wait for the central node's request response before executing a task, which makes it take longer for other nodes to execute tasks and reduces processing efficiency. The existence of a central node generally requires a separate server, which requires research and development investment in memory, CPU, hard disk, etc., and maintenance costs also increase accordingly.

[0052] If the centralized node is removed, the system does not rely on a single node and can respond faster because the node can process requests locally without waiting for instructions from the central node. Since there is no need to maintain expensive central nodes, the R&D cost is lower than that of centralized systems. The decentralized system does not rely on a single node, so the failure of any node will not cause the entire system to crash, and the stability is higher. However, the removal of centralized nodes has caused deficiencies in the system in terms of data control and data management.

[0053] Taking the above into consideration, how to accurately control and manage data while removing centralized nodes is the main technical problem to be solved by this application.

[0054] For example, multiple task nodes including Node A, Node B, and Node C are used as a decentralized distributed architecture for processing. During the processing, the order of each node needs to be ensured, and the data needs to be retrieved accurately to ensure that no data is missed and cannot be processed repeatedly.

[0055] Reference Manual Attached Figure 1 An embodiment of a decentralized multi-node same-data source data distribution method provided by the present application includes the following steps:

[0056] S100, multiple task nodes compete for initialization authority, and a task node that obtains the initialization authority is used as an initialization node.

[0057] Specifically, at the beginning of a task, multiple nodes compete for initialization permissions through the memory distributed lock in redis. Redis is an open source, high-performance key-value database. It supports different types of data structure services. Redis provides high-performance memory data structure storage, making it very suitable for use as a message queue, and can efficiently manage the queuing and processing of multiple tasks.

[0058] After the competition, a certain node has the initialization permission. Optionally, during the competition, it may be affected by the network, regional area, the temperature of the machine at that time, CPU, memory usage, etc. It can be set that the task node that arrives in the memory database first has priority to obtain the initialization permission. In other implementations, special conditions can also be set to filter the task nodes. In the Redis cluster, special conditions are set such as the overall status of the node at that time, or the task node with the best network status, the lowest temperature, the least CPU occupancy, and the least memory occupancy has priority to compete for initialization permission. The method of competing for initialization permission is not specifically limited in this application.

[0059] S200, after obtaining the initialization authority, the initialization node obtains a data source from a cloud server, stores the data source in a memory database, and performs initialization processing on the data source to generate a data queue.

[0060] Specifically, a node with initialization authority initializes the data by putting all the data to be distributed into a three-party queue. In one implementation, Rqueue in redis is used as the three-party queue. Rqueue is a distributed job queue system based on Redis, which allows developers to implement asynchronous task processing and planned task scheduling in applications. Since Redis provides high-performance memory data structure storage, Rqueue can quickly store and retrieve tasks in the queue. In other implementations of this embodiment, other existing queue tools can also be used as three-party queues. This application does not make specific restrictions.

[0061] S300, after completing the initialization task, the initialization node also serves as the task node, extracting data units from the data queue for processing in a specified order.

[0062] Each node extracts a data unit from the queue each time and processes it. In this embodiment, processing refers to executing a specific business, such as presetting the execution task to distribute gift certificates. Then the processing here is to get the unique identity code of each user's mobile phone. To send gift certificates, users can receive gift certificate notifications from merchants.

[0063] Another embodiment of a decentralized multi-node same-data-source data distribution method of the present application, based on an embodiment of the above method, step S100 includes:

[0064] S110, before requesting permission, the task node verifies the node status to ensure that it meets the conditions for participating in the initialization permission competition.

[0065] Specifically, a node needs to pass identity verification to confirm its identity and determine whether it has the conditions to perform tasks.

[0066] S120, when the competition starts, the task node sends a request to the server to obtain a distributed lock according to the competition mechanism to ensure exclusive access to the data source.

[0067] Specifically, a distributed lock is a mutual exclusion mechanism used in a distributed system to ensure that multiple nodes access shared resources at the same time. It can ensure that only one node can access shared resources at any time, and other nodes need to wait.

[0068] S130: If any task node is successfully acquired, the competition result is returned.

[0069] S140: If the task node does not receive a response within the specified time, the task node times out and retries the request to obtain the distributed lock.

[0070] For details, please refer to the attached manual. Figure 2 As shown, nodes A, B, and C compete for initialization authority, and the node that first obtains a response, such as node A, obtains the initialization authority.

[0071] Optionally, the node that obtains permission needs to synchronize its status with other nodes after the operation to maintain system consistency. If all nodes do not obtain permission or receive no response within a certain period of time, it may time out and re-initiate a request or back off to try to obtain permission again.

[0072] In one implementation of this embodiment, optionally, multiple nodes compete for initialization authority, and determine the node that obtains the initialization authority based on the number of node processes, hardware load, temperature, network, etc. In some other implementations of this embodiment, multiple nodes compete for initialization authority, which is implemented based on the CSMA / CA mechanism. CSMA / CA is a commonly used channel access control mechanism. Before sending data, the node will first listen to whether the channel is idle. If the channel is idle, it will enter the backoff process and select a random number from the contention window as the backoff time. Each node will listen to the channel during the backoff process. When the backoff timing ends, if the channel is still idle, the node can send data. This mechanism reduces the possibility of channel conflicts by random backoff. Or it can be implemented according to the priority channel competition method to ensure that high-priority data streams can obtain channel access rights faster. More preferably, other existing competition methods can also be used to implement node authority competition.

[0073] The present application discloses another embodiment of a decentralized multi-node same data source data distribution method. Based on the above embodiment of the method, refer to the attached specification. Figure 3 As shown, the step S200 includes:

[0074] S210: After obtaining the initialization authority, the initialization node obtains a data source from a cloud server and stores the data source in a memory database.

[0075] For example, if node A obtains the initialization permission, the cloud server needs to obtain the data source and place it in the memory database. The data source is all the data required for this data distribution task.

[0076] S220, after receiving the data source, the memory database traverses the data source and organizes all the data therein into the data queue.

[0077] Specifically, after the memory database receives the data source, the internal function traverses the data source and generates an Rqueue queue. The traversal of node A is as follows: first, the data source of node A is placed in the memory database, and then the initialization flag is determined. If there is no initialization flag, the initialization flag is set first and then the data is extracted and processed from the queue. The traversal of nodes B and C is the same. The initialization flag is determined first. At this time, because the operation of node A has determined the initialization flag, nodes B and C return to the queue normally to extract and process data.

[0078] S230, dividing the data queue according to a preset data amount to obtain a plurality of data units. The initialization node increases a number for each of the data units and defines an initialization flag.

[0079] Specifically, in some implementations of this embodiment, the data source is cut according to the data situation, and the data source is cut into multiple parts. Optionally, equal cutting and unequal cutting can be performed. The specific situation can be cut according to the processing capacity of a single node. In one implementation of this embodiment, equal cutting is performed by default. For example, if there are 100,000 pieces of data in each copy, node A numbers each data unit, such as ①: 1-100000. ②: 100001-200000. ... ④: 300001-400000... And define an initialization flag bit for each data unit. The purpose of defining the initialization flag bit is to ensure that other nodes do not re-initialize the above queues during the subsequent data extraction process.

[0080] Another embodiment of a decentralized multi-node same-data-source data distribution method of the present application, based on an embodiment of the above method, step S300 includes the following steps:

[0081] S310, each of the task nodes extracts one data unit at a time for processing.

[0082] S320: After the data unit is extracted, the memory database deletes the data unit.

[0083] For details, please refer to the attached manual. Figure 4 As shown, after completing the initialization task, node A, as the initialization node, performs data processing tasks together with other nodes B and C.

[0084] For example, the maximum data processing capacity of a single node is defined as 100,000. Each node extracts a piece of data from the queue each time, processes and sends it. After each node extracts the data it needs to process from the data point, it deletes the extracted data through the Rqueue queue of the memory database (delete after reading), ensuring that each node will not extract old data each time it extracts new data, ensuring accurate and non-duplicate processing.

[0085] Another embodiment of a decentralized multi-node same-data-source data distribution method of the present application, based on an embodiment of the above method, further includes the following steps:

[0086] S330, after the current data unit is processed, the task node re-extracts the next data unit until the data value of the data queue is zero.

[0087] Optionally, it also includes: S340, after the data processing is completed, the node that last processes the data deletes the initialization flag.

[0088] For details, please refer to the attached manual. Figure 5As shown. After the node completes the processing of the currently taken data, it returns to the task queue to take new data again until all data processing is completed and the amount of data in the data queue becomes zero. After each node completes data processing, it determines whether the value of the next data in the queue is zero. If it is not zero, it extracts new data for processing and distribution. During the continuous processing, when the last data is processed, the initialization flag defined in step S230 is deleted and the next command is waited for. This prevents the subsequent task of this type from failing to initialize when it starts.

[0089] Another embodiment of the method of the present application, based on any of the above embodiments, the configuration of the initialization node is the same as that of the task node. On the basis of the general task node, the initialization configuration is added. Using the general task node as the initialization node, the R&D cost is relatively centralized, and the development of the server is reduced. Only the working node needs to be modified. The amount of changes is small. This solution only modifies the original node. And only the memory database is used, with low economic cost. During maintenance, only the original node needs to be maintained.

[0090] Based on the same technical concept, the present application also discloses a decentralized multi-node same-data source data distribution system, which can be used to implement any of the above-mentioned decentralized multi-node same-data source data distribution methods. Specifically, an embodiment of a decentralized multi-node same-data source data distribution system of the present application includes multiple task nodes, a cloud server, and an in-memory database.

[0091] The plurality of task nodes are used to compete for initialization authority, and a task node that competes to obtain the initialization authority serves as an initialization node.

[0092] The initialization node is used to obtain the data source from the cloud server after obtaining the initialization authority, store the data source in the memory database, and perform initialization processing on the data source to generate a data queue.

[0093] After completing the initialization task, the initialization node also serves as the task node, extracting data units from the data queue for processing in a specified order.

[0094] Another embodiment of a decentralized multi-node same-data source data distribution system provided by the present application is, based on the above system embodiment, after the in-memory database receives the data source, it is also used to traverse the data source and organize all the data therein into the data queue.

[0095] The memory database is also used to divide the data queue according to a preset data volume to obtain a plurality of data units.

[0096] The initialization node is further used to add a number to each of the data units and define an initialization flag.

[0097] Preferably, each of the task nodes extracts one data unit at a time for processing.

[0098] After the data unit is extracted, the memory database deletes the data unit.

[0099] Preferably, after the current data unit is processed, the task node re-extracts the next data unit until the data value of the data queue is zero.

[0100] After the data processing is completed, the node that last processes the data deletes the initialization flag.

[0101] Another embodiment of a decentralized multi-node same-data source data distribution system provided by the present application is based on the above system embodiment. Before requesting permission, the task node verifies the node status to ensure that it meets the conditions for participating in the initialization permission competition.

[0102] When competition begins, the task node sends a request to the server to obtain a distributed lock according to the competition mechanism to ensure exclusive access to the data source.

[0103] If any task node is successfully acquired, the competition result is returned.

[0104] If the task node does not receive a response within the specified time, it times out and retries the request to obtain the distributed lock.

[0105] Preferably, the configuration of the initialization node is the same as that of the task node.

[0106] The decentralized multi-node same-data source data distribution method and system of the present application have the same technical concept, and the technical details of the two embodiments are applicable to each other. In order to reduce repetition, they will not be repeated here.

[0107] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned program modules is used as an example for illustration. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program units or modules to complete all or part of the functions described above. The program modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into a processing unit, and the above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software program unit. In addition, the specific names of the program modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0108] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

Claims

1. A decentralized multi-node same-data source data distribution method, characterized in that: The steps include: A plurality of task nodes compete for initialization authority, and a task node that obtains the initialization authority is used as an initialization node; After obtaining the initialization authority, the initialization node obtains the data source from the cloud server and stores the data source in the memory database; and performs initialization processing on the data source to generate a data queue; After completing the initialization task, the initialization node also serves as the task node, extracting data units from the data queue for processing in a specified order.

2. A decentralized multi-node same-data source data distribution method as claimed in claim 1, characterized in that: The initialization process of the data source to generate a data queue comprises the following steps: After receiving the data source, the memory database traverses the data source and organizes all the data therein into the data queue; The data queue is divided according to a preset data amount to obtain a plurality of data units; the initialization node increases a number for each of the data units and defines an initialization flag.

3. A decentralized multi-node same-data source data distribution method as claimed in claim 1, characterized in that: The extracting of data units from the data queue for processing in a specified order comprises the following steps: Each of the task nodes extracts one data unit at a time for processing; After the data unit is extracted, the memory database deletes the data unit.

4. A decentralized multi-node same-data source data distribution method as claimed in claim 1, characterized in that: Also includes: After the current data unit is processed, the task node re-extracts the next data unit until the data value of the data queue is zero.

5. A decentralized multi-node same-data source data distribution method as claimed in claim 2, characterized in that: Also includes: After the data processing is completed, the node that last processes the data deletes the initialization flag.

6. A decentralized multi-node same-data source data distribution method as claimed in claim 1, characterized in that: The multiple task nodes compete for initialization authority, and one of the task nodes that obtains the initialization authority is used as the initialization node; the steps include: Before requesting permission, the task node verifies the node status to ensure that it meets the conditions for participating in the initialization permission competition; When competition starts, the task node sends a request to the server to obtain a distributed lock according to the competition mechanism to ensure exclusive access to the data source; If any task node is successfully acquired, the competition result is returned; If the task node does not receive a response within the specified time, it times out and retries the request to obtain the distributed lock.

7. A decentralized multi-node same-data source data distribution method as claimed in any one of claims 1 to 6, characterized in that: The configuration of the initialization node is the same as that of the task node.

8. A decentralized multi-node same-data source data distribution system, comprising multiple task nodes, a cloud server, and a memory database; characterized in that: The system is used to execute a decentralized multi-node same-data source data distribution method as described in any one of claims 1-7.

9. A decentralized multi-node same-data-source data distribution system as claimed in claim 8, characterized in that: A plurality of the task nodes are used to compete for initialization authority, and a task node that competes to obtain the initialization authority serves as an initialization node; The initialization node is used to obtain the data source from the cloud server after obtaining the initialization authority, store the data source in the memory database; and perform initialization processing on the data source to generate a data queue; After completing the initialization task, the initialization node also serves as the task node, extracting data units from the data queue for processing in a specified order.

10. A decentralized multi-node same-data-source data distribution system as claimed in claim 8, characterized in that: After receiving the data source, the memory database is further used to traverse the data source and organize all the data therein into the data queue; The memory database is further used to divide the data queue according to a preset data volume to obtain a plurality of data units; The initialization node is further used to add a number to each of the data units and define an initialization flag.

Images