Distributed big data processing task scheduling method for non-peer communication

Abstract

The invention discloses an asymmetrical communication-oriented distributed big data processing task scheduling method, which comprises the following steps that: when a processing round starts, a control node distributes latest to-be-processed data and a model to a computing node required by the round of processing according to a current environmental condition, and the computing node receives the latest to-be-processed data and the model and then sends the latest to-be-processed data and the model to the computing node; after multiple rounds of data processing are executed according to the processing round information of the control node, a processing result is sent to the control node, the control node carries out negative influence monitoring on the received processing result, if the negative influence exists, the processing result of the computing node is abandoned, and the aggregation weight of the processing result which is not abandoned by each computing node is determined. And the control node aggregates the processing results of different computing nodes according to the aggregation weight of each node, and then performs the next round of processing according to the current environmental condition. According to the invention, high-efficiency distributed big data processing can be realized in a network environment with relatively large uplink and downlink bandwidth difference, and the overall processing efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of big data processing and parallel computing, and in particular to a distributed big data processing task scheduling method for non-peer communication. Background Art

[0002] Distributed big data processing refers to improving the efficiency and speed of data processing by distributing data to multiple computing nodes or servers and using parallel processing technology. With the rapid development of the Internet, the Internet of Things, social media and mobile devices, the amount of global data has exploded. Traditional centralized data processing methods have been unable to cope with such a large amount of data due to the limited computing power and storage capacity of a single server. In this situation, distributed big data processing came into being.

[0003] The basic idea of ​​distributed processing is to decompose big data tasks into several smaller tasks. By parallelizing these tasks, the processing time can be significantly shortened, and the storage and computing resources of multiple machines can be effectively utilized. For example, in order to meet the inference class requests of computing users, it is necessary to deploy a certain number of instances for each version of the inference class model in each resource-constrained computing node. These model instances have the same functions and can process the requests of computing users concurrently. During the data processing process, the control node will continuously update the instances and parameters according to the processing results of each computing node, and send the updated instances and parameters to these computing nodes. In order to reduce the delay between the control node and each computing node, it is more appropriate to deploy the control node in a computing area close to a large number of computing nodes than in a data center.

[0004] However, for control nodes that need to communicate with a large number of computing nodes at the same time, the current symmetric communication method will cause its communication upper limit to be restricted by a smaller uplink bandwidth, resulting in a small number of computing nodes that the control node can communicate with in each round to send instances and requirements, which wastes a lot of computing node computing power resources and a lot of downlink bandwidth.

[0005] For this reason, there is an urgent need for a new strategy to solve this problem, to increase the downlink bandwidth of control nodes and the utilization rate of computing resources of computing nodes, and to improve the efficiency of distributed big data processing. Summary of the invention

[0006] The purpose of the present invention is to provide a distributed big data processing task scheduling method for non-peer communication. By combining an asymmetric communication strategy and a distributed big data processing method, task scheduling is dynamically adjusted according to task requirements, the status of control nodes, network bandwidth and other conditions, so as to solve the problem of waste of control node downlink bandwidth and computing node computing power resources in the current symmetric communication distributed big data processing task method.

[0007] In order to solve the above technical problems, the present invention provides the following technical solution: a distributed big data processing task scheduling method for non-peer communication, wherein the scheduling method steps include:

[0008] At the beginning of a processing round, the control node distributes the latest data to be processed and the model to the computing nodes required for this round of processing according to the current environmental conditions, so as to perform local calculations and model updates; the environmental conditions include but are not limited to the current processing requirements, computing node status and network bandwidth conditions;

[0009] After receiving the latest data and model to be processed, the selected computing node in this round performs multiple rounds of data processing according to the processing round information of the control node; the control node and the computing node communicate with each other using an asymmetric communication mechanism;

[0010] After each round of data processing is completed, the computing node sends the processing results to the control node. If the computing node has not received new data and models to be processed after processing the rounds specified by the control node, it will continue to wait;

[0011] After obtaining the processing results in each round, the control node monitors the processing results transmitted by each computing node. If it is detected that the processing results transmitted by a computing node have a negative impact on the overall result, the processing results transmitted by the computing node are discarded; wherein, the processing results transmitted by each computing node can be monitored by methods such as consistency protocol, business logic verification and clustering based on machine learning. If the deviation of a certain result is greater than or equal to a preset threshold, it means that the processing result transmitted by the computing node has a negative impact on the overall result;

[0012] After discarding negative results, the control node will determine the aggregation weight of the processing results not discarded by each computing node according to the constraints of the computing nodes, and aggregate the results processed by different computing nodes according to the aggregation weights of each node; the constraints include but are not limited to the quality of computing node data, the time since the last communication, the amount of computing node data, and the difference between the current round results of the computing node and the round when the latest processing model was obtained last time;

[0013] After aggregating the results processed by different computing nodes, the control node re-determines the computing nodes to be selected in the next round based on the current environmental conditions.

[0014] According to the above technical solution, the computing node is deployed in an asymmetric network where the uplink bandwidth is smaller than the downlink bandwidth, and is used to perform multiple rounds of data processing and return the results to the control node after each round of processing;

[0015] The control node is deployed in an asymmetric network environment adjacent to the computing node, and is responsible for coordinating and monitoring the entire computing network; the control node schedules different computing nodes based on factors such as the current processing requirements, computing node status, and network bandwidth conditions, and determines the next round of selection. At the same time, after obtaining the results of each round of processing, the control node determines the aggregation weight of the results of each computing node based on factors such as the amount of data and communication delay of the computing node, and aggregates the results. Since computing nodes are often deployed in an asymmetric network where the uplink bandwidth is smaller than the downlink bandwidth, and in order to reduce network delays, control nodes are usually also deployed in adjacent asymmetric network environments.

[0016] The asymmetric communication mechanism is that after the control node transmits the processing model to the computing node once, the computing node transmits the processing results back to the control node multiple times. Since the control node is often deployed in an asymmetric network environment where the uplink bandwidth is smaller than the downlink bandwidth, for the control node that needs to communicate with a large number of computing nodes at the same time, the use of symmetric communication will cause its communication upper limit to be limited by the smaller uplink bandwidth, thereby wasting a large amount of downlink bandwidth resources and the computing power of the computing node. Therefore, the use of an asymmetric communication mechanism enables the control node to maximize the use of the downlink bandwidth under limited uplink bandwidth conditions and download processing results from multiple computing nodes.

[0017] Traditional distributed processing methods usually use single-round communication processing, that is, the computing node sends the results back to the control node after each round of processing, and the control node adjusts the processing model and parameters according to the results, and then sends them to the computing node for the next round of processing. In the present invention, based on the asymmetric communication strategy, the computing node will perform multiple rounds of processing after receiving the processing model and data once, and return the processing results to the control node multiple times. In this process, the computing node will adaptively modify the model according to its own processing results, and each round of processing is based on the model modified in the previous round.

[0018] According to the above technical solution, when determining the weight of the results from different computing nodes when aggregating, two factors are considered: the amount of computing node data and the difference between the current round result of the computing node and the round when the latest processing model was obtained last time. The round of data processing performed by each computing node may be different.

[0019] According to the above technical solution, at the beginning of a processing round, the control node first calculates the Gaussian weight of each node according to the network conditions and computing power of the computing node, and then calculates the probability of each computing node being selected in this round of processing according to the Gaussian weight. i :

[0020]

[0021] In the formula, represents the processing time of computing node i, represents the communication duration of computing node i, represents the processing time of computing node j, represents the communication duration of computing node j, and σ represents the scale parameter of Gaussian weight control attenuation.

[0022] The larger the Gaussian weight of a node, the more beneficial the node is to the overall data processing (the shorter the sum of processing and communication time, the larger the weight). Here, the normalized Gaussian weight is used as the probability of selection, and then sampling is performed based on this probability to obtain the computing node selected in this round. The method of calculating the probability first and then sampling can avoid dependence on specific clients and improve the robustness of the system. Gaussian weighting can avoid excessive differences in probability between different clients.

[0023] The processing results of each round may include data generated by the computing node based on the latest model, or may include results based on outdated models. Therefore, when the control node detects that the results transmitted by a computing node have a negative impact on the overall results, the control node will discard the data. Subsequently, the control node decides whether to immediately transmit the latest processing model to the computing node or temporarily stop the node's task based on the computing node selection algorithm to optimize the overall processing efficiency.

[0024] According to the above technical solution, when the results are aggregated in the rth round, the aggregation weight q of node i is calculated. r,i :

[0025]

[0026] In the formula, Indicates the node data volume weight of computing node number i in round r, Indicates the node round difference weight of the calculation node number i in the rth round, k r Indicates the number of computing nodes participating in the result aggregation in round r, n indicates the amount of data, Indicates the node data volume weight of computing node number j in round r, Indicates the node round difference weight of the node number j in the rth round.

[0027] The control node dynamically adjusts the aggregation weight of each node's processing results based on factors such as the amount of data and communication delay of each computing node. For computing nodes with higher data quality and shorter communication intervals with the control node, the control node will assign higher weights to enhance the impact of their results on the final aggregation results, thereby more effectively correcting the global processing results.

[0028] According to the above technical solution, the node data volume weight

[0029]

[0030] In the formula, k r Indicates the number of computing nodes participating in the result aggregation in round r, n indicates the amount of data, and n i represents the amount of data in computing node i, n j Indicates the amount of data for node j.

[0031] According to the above technical solution, the node round difference weight

[0032]

[0033] Where Δr represents the round difference between the round corresponding to the result of the current round of the computing node and the round when the latest processing model was obtained last time, and σ represents the scale parameter of the Gaussian weight control attenuation.

[0034] According to the above technical solution, the number of processing rounds executed by the computing node is limited by both the downlink bandwidth of the control node and the resources of the computing node.

[0035] Since computing nodes have limited resources, data processing can only be performed within the range that their resources can bear. The control node must not only use the computing node selection algorithm to decide whether to select a computing node to participate in the processing based on factors such as the computing node's processing power, network latency, and data value, but also needs to further reasonably allocate the processing rounds of the node to ensure that its task load does not exceed the resource tolerance range, thereby ensuring the efficient operation of the overall system. That is, the control node will accumulate multiple rounds of processing instructions issued to the computing node to ensure that the results received in each round will not exceed the upper limit of the downlink bandwidth. Based on this restriction, the computing node will perform the most rounds of processing within the range that its own resources can bear. This ensures that the computing power resources of the computing node are fully utilized and that the results are not wasted due to bandwidth limitations.

[0036] Another embodiment provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program runs on the processor, it executes any one of the technical solutions of the above-mentioned distributed big data processing task scheduling method for non-peer communication.

[0037] Another embodiment is also included, a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, it implements any one of the technical solutions of the above-mentioned distributed big data processing task scheduling method for non-peer communication.

[0038] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0039] 1. The present invention proposes an asymmetric communication mechanism, which is suitable for asymmetric scenarios where the network uplink bandwidth of the control node is smaller than the downlink bandwidth. The mechanism fully considers the asymmetry of the network uplink and downlink bandwidths, especially when the uplink bandwidth is limited, and designs a solution that can fully utilize the downlink bandwidth. Through this mechanism, the present invention effectively avoids the waste of bandwidth resources, ensures that data processing tasks can be run in an efficient and stable manner, and maintains the low-cost advantage of the overall network environment. This design does not require additional hardware costs, significantly optimizes data transmission efficiency, and provides a strong guarantee for the smooth completion of long-term big data tasks.

[0040] 2. In the present invention, after receiving a processing model and data, the computing node will perform multiple rounds of processing and return the processing results to the control node multiple times. This allows the computing node to process and infer data based on the historical model even if it fails to receive the latest processing model. This not only ensures the efficient operation of the asymmetric communication mechanism, but also makes full use of the computing power resources of the computing nodes to avoid idle resources. In addition, this mechanism significantly enhances the autonomy and robustness of the computing nodes, enabling them to efficiently perform tasks under complex network conditions and maximize their effectiveness.

[0041] 3. In the present invention, the control node dynamically adjusts the aggregate weight of the processing results of each node according to the data volume, communication delay and other factors of each computing node, and performs resource management and allocation on the computing node through the computing node selection algorithm. The aggregate weight of the processing results can effectively integrate the intermediate results generated by each computing node in the case of multi-node parallel processing, ensure the accuracy and consistency of the data, and maximize the processing results close to the ideal model. The computing node selection algorithm significantly improves the overall processing efficiency by balancing the load of each node and avoiding node overload or idleness. Ensure that the computing power resources of the computing nodes are fully utilized, while avoiding the waste of result transmission due to bandwidth limitations, and optimize the overall resource allocation and utilization of the system.

[0042] 4. By combining asymmetric communication strategies and distributed big data processing methods, the present invention enables the system to dynamically adjust task scheduling according to task requirements, control node conditions, and network bandwidth conditions, thereby avoiding waste of bandwidth and computing resources and improving overall processing efficiency. It can achieve efficient distributed big data processing in a network environment with large differences in upstream and downstream bandwidths, thereby maximizing the system processing capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0044] Figure 1It is a schematic diagram of a big data processing method using the distributed big data processing task scheduling method for non-peer communication of the present invention;

[0045] Figure 2 This is a schematic diagram of traditional distributed big data processing methods. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] The present invention proposes an asymmetric communication mechanism such as Figure 1 As shown, after the control node transmits a processing model to the computing node once, the computing node will perform multiple rounds of processing after receiving the processing model and settings once, and return the processing results to the control node multiple times. Figure 1 At the beginning of the processing round, the control node selects computing nodes 1 and 2 according to the current environmental conditions in the first round and sends the model to them for processing. Computing nodes 4 and 5 process according to their own local models. After computing nodes 1, 4 and 5 have completed multiple rounds of data processing, the processing results and models are sent to the control node. The control node selects computing nodes 3 and 4 according to the current environmental conditions and sends the model to them for processing. Computing nodes 1 and 2 process according to their own local models. Computing node 5 stops processing because it has not received the model for a long time. After computing nodes 1, 3 and 4 have completed multiple rounds of data processing, the processing results and models are sent to the control node. The control node performs the third round of processing according to the current environmental conditions, and this cycle continues.

[0048] In the prior art, since control nodes are often deployed in an asymmetric network environment where the uplink bandwidth is smaller than the downlink bandwidth, for control nodes that need to communicate with a large number of computing nodes at the same time, symmetric communication is used. Figure 2 As shown in the figure, the computing node sends the results back to the control node after each round of processing. The control node adjusts the processing model and parameters based on the results and then sends them to the computing node for the next round of processing. Figure 2In the process, at the beginning of the processing round, the control node selects computing nodes 1 and 2 according to the current environmental conditions and sends the model to them for processing. The other computing nodes are in an idle state. After the first round of computing nodes 1 and 2 are processed, the processing results and the model are sent to the control node. The control node selects computing nodes 3 and 4 according to the current environmental conditions and sends the model to them for the second round of processing. The other nodes are in an idle state. After the second round of computing nodes 3 and 4 are processed, the processing results and the model are sent to the control node. The control node selects computing nodes 1 and 5 according to the current environmental conditions and sends the model to them for the third round of processing. After the third round of computing nodes 1 and 5 are processed, the processing results are sent to the control node. The control node adjusts the selection of the next round of computing nodes for data processing according to the processing results of the third round, and the cycle continues until the data processing is completed.

[0049] By the above Figure 1 and Figure 2 It can be seen that in the prior art, since the upper limit of symmetric communication is limited by a small uplink bandwidth, the number of computing nodes that can communicate with each control node to send instances and requirements is small, which wastes a lot of computing node computing resources and a lot of downlink bandwidth. The asymmetric communication mechanism of the present invention can ensure that the computing resources of the computing nodes are fully utilized, and also ensure that the results will not be wasted due to bandwidth limitations.

[0050] The present invention provides a technical solution, a distributed big data processing task scheduling method for non-peer communication, the steps of which include:

[0051] S1. At the beginning of a processing round, the control node distributes the latest data to be processed and models to the computing nodes required for this round of processing based on the current environmental conditions, so as to perform local calculations and model updates; environmental conditions include but are not limited to the current processing requirements, computing node status and network bandwidth conditions.

[0052] That is, the calculation node selection algorithm uses the processing time To quantify the processing power of the computing node itself, use the communication duration To quantify the network delay, the total time to complete the processing of each round of selecting the computing node is Calculate the Gaussian weight of the node, and then calculate the probability of the node being selected according to the Gaussian weight of the node; calculate the probability of node i being selected as p i :

[0053]

[0054] In the formula, represents the processing time of computing node i, represents the communication duration of computing node i, represents the processing time of computing node j, represents the communication duration of computing node j, and σ represents the scale parameter of Gaussian weight control attenuation.

[0055] The number of processing rounds executed by the computing node is limited by the downlink bandwidth of the control node and the resources of the computing node. In general, the control node will accumulate the multiple rounds of processing instructions it sends to the computing node to ensure that the results received in each round do not exceed the upper limit of the downlink bandwidth.

[0056] S2. After receiving the latest data and model to be processed, the selected computing node in this round performs multiple rounds of data processing according to the processing round information of the control node; an asymmetric communication mechanism is used for communication between the control node and the computing node; among which, the rounds of data processing performed by different computing nodes may also be different.

[0057] S3. After each round of data processing is completed, the computing node sends the processing results to the control node. If the computing node has not received new data and models to be processed after processing the rounds specified by the control node, it will continue to wait;

[0058] S4. After obtaining the processing results in each round, the control node will first use the consistency protocol, business logic verification, and machine learning-based clustering methods to monitor the processing results transmitted by each computing node. If it is detected that the processing result transmitted by a computing node is greater than the set threshold P, the processing result transmitted by the computing node is discarded;

[0059] S5. After discarding the negative results, the control node will determine the aggregation weight of the processing results not discarded by each computing node according to the constraints of the computing nodes, and aggregate the results processed by different computing nodes according to the aggregation weights of each node; the constraints include but are not limited to the quality of computing node data, the length of time since the last communication, the amount of computing node data, and the difference between the current round results of the computing node and the last round of obtaining the latest processing model;

[0060] That is, when the results are aggregated in the rth round, the aggregation weight q of node i is calculated r,i :

[0061] In the formula, Indicates the node data volume weight of computing node number i in round r, Indicates the node round difference weight of the calculation node number i in the rth round, k r Indicates the number of computing nodes participating in the result aggregation in round r, n indicates the amount of data, Indicates the node data volume weight of computing node number j in round r, Indicates the node round difference weight of the node number j in the rth round.

[0062] Among them, the node data volume weight

[0063]

[0064] In the formula, k r Indicates the number of computing nodes participating in the result aggregation in round r, n indicates the amount of data, and n i represents the amount of data in computing node i, n j Indicates the amount of data for node j.

[0065] Node round difference weight

[0066]

[0067] Where Δr represents the round difference between the round corresponding to the result of the current round of the computing node and the round when the latest processing model was obtained last time, and σ represents the scale parameter of the Gaussian weight control attenuation.

[0068] S6. After aggregating the results processed by different computing nodes, the control node re-determines the computing nodes to be selected in the next round based on the current environmental conditions.

[0069] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0070] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A distributed big data processing task scheduling method for non-peer communication, characterized in that: The scheduling method comprises: At the beginning of a processing round, the control node distributes the latest data to be processed and the model to the computing nodes required for this round of processing according to the current environmental conditions, so as to perform local calculations and model updates; the environmental conditions include the current processing requirements, computing node status and network bandwidth conditions; After receiving the latest data and model to be processed, the selected computing node in this round performs multiple rounds of data processing according to the processing round information of the control node, and the control node and the computing node communicate with each other using an asymmetric communication mechanism; After each round of data processing is completed, the computing node sends the processing results to the control node. If the computing node has not received new data and models to be processed after processing the rounds specified by the control node, it continues to wait; After obtaining the processing results in each round, the control node monitors the processing results transmitted by each computing node. If it is detected that the processing results transmitted by a computing node have a negative impact on the overall results, the processing results transmitted by the computing node are discarded. After discarding negative results, the control node will determine the aggregation weight of the processing results not discarded by each computing node according to the constraints of the computing nodes, and aggregate the results processed by different computing nodes according to the aggregation weight of each node; the constraints include the quality of computing node data, the time since the last communication, the amount of computing node data, and the difference between the current round result of the computing node and the round when the latest processing model was obtained last time; After aggregating the results processed by different computing nodes, the control node re-determines the computing nodes to be selected in the next round based on the current environmental conditions.

2. The distributed big data processing task scheduling method for non-peer communication according to claim 1 is characterized in that: The computing node is deployed in an asymmetric network where the uplink bandwidth is smaller than the downlink bandwidth, and is used to perform multiple rounds of data processing and return the results to the control node after each round of processing; The control node is deployed in an asymmetric network environment adjacent to the computing node and is responsible for coordinating and monitoring the entire computing network; The asymmetric communication mechanism is that after the control node transmits a processing model to the computing node once, the computing node transmits the processing results back to the control node multiple times.

3. The distributed big data processing task scheduling method for non-peer communication according to claim 1 is characterized in that: The rounds of data processing performed by each computing node may be different.

4. The distributed big data processing task scheduling method for non-peer communication according to claim 1 is characterized in that: At the beginning of a processing round, the control node first calculates the Gaussian weight of each node according to the network conditions and computing power of the computing node, and then calculates the probability of each computing node being selected in this round of processing according to the Gaussian weight. The probability of selecting the computing node i is p i : In the formula, represents the processing time of computing node i, represents the communication duration of computing node i, represents the processing time of computing node j, represents the communication duration of computing node j, σ represents the scale parameter of Gaussian weight control attenuation, and i, j and k are natural numbers.

5. The distributed big data processing task scheduling method for non-peer communication according to claim 1 is characterized in that: After the control node discards the negative results in the rth round, it determines the aggregate weight q of the processing results not discarded by each computing node according to the constraints of the computing node i. r,i : In the formula, Indicates the node data volume weight of computing node number i in round r, Indicates the node round difference weight of the calculation node number i in the rth round, k r Indicates the number of computing nodes participating in the result aggregation in round r, n indicates the amount of data, Indicates the node data volume weight of computing node number j in round r, Indicates the node round difference weight of the node number j in the rth round.

6. The distributed big data processing task scheduling method for non-peer communication according to claim 5 is characterized in that: The node data volume weight It is expressed by the following formula: In the formula, k r Indicates the number of computing nodes participating in the result aggregation in round r, n indicates the amount of data, and n i represents the amount of data in computing node i, n j Indicates the amount of data for node j.

7. The distributed big data processing task scheduling method for non-peer communication according to claim 5 is characterized in that: The node round difference weight Where Δr represents the round difference between the round corresponding to the result of the current round of the computing node and the round when the latest processing model was obtained last time, and σ represents the scale parameter of the Gaussian weight control attenuation.

8. The distributed big data processing task scheduling method for non-peer communication according to claim 1 is characterized in that: The number of processing rounds executed by the computing node is limited by both the downlink bandwidth of the control node and the resources of the computing node.

9. An electronic device, characterized in that: It includes a memory and a processor, the memory stores a computer program, and when the computer program runs on the processor, it executes the distributed big data processing task scheduling method for non-peer communication described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the distributed big data processing task scheduling method for non-peer communication according to any one of claims 1 to 8.

Images