Self-adaptive parallel data transmission method and system

Abstract

The invention discloses a self-adaptive parallel data transmission method and system, and relates to the technical field of computer networks and data processing. Dividing to-be-transmitted data to obtain a plurality of sub-data blocks, and summarizing the sub-data blocks into a global task list; screening out an available server set in an idle state from the server cluster; calculating an aggregation transmission rate in the current time window; if the current aggregation transmission rate is greater than or equal to the historical aggregation transmission rate, increasing the current degree of parallelism, and adding a new target server from the available server set; if the current aggregation transmission rate is smaller than the historical aggregation transmission rate, the current parallelism degree is reduced, and the server is released from the current target server; and if all the sub-data blocks in the global task list are successfully transmitted, splicing and checking the target sub-data blocks, and ending the data transmission after the target sub-data blocks are checked to be correct. By implementing the provided technical scheme, the problem that an existing fixed concurrent transmission scheme cannot adapt to environmental changes is solved.

Description

Technical Field

[0001] This application relates to the field of computer networks and data processing technology, specifically to an adaptive parallel data transmission method and system. Background Technology

[0002] In the wave of enterprise digital transformation, data migration has become a fundamental and critical IT activity, widely used in scenarios such as enterprise cloud migration, data center consolidation, disaster recovery system construction, and system upgrades. These migration tasks often involve massive amounts of data at the TB or even PB level. How to quickly, securely, and reliably complete the migration of data from the source to the target within the preset migration window is directly related to business continuity and the success or failure of the project.

[0003] To address the challenges of large-scale data migration, existing technologies generally employ parallel transmission mechanisms. Before migration begins, this mechanism divides the source data assets into multiple data blocks using a specific algorithm and pushes these blocks simultaneously to the target system based on a pre-defined fixed number of concurrent streams. This approach fully utilizes the network bandwidth and I / O processing capabilities between the source and target systems, aiming to minimize the overall migration time.

[0004] However, the strategies mentioned above that rely on a fixed number of concurrent streams cannot dynamically adapt to real-time changes across the entire migration process. Data migration typically involves complex network environments (such as public networks and leased lines) and shared storage and computing resources, whose available bandwidth, latency, and I / O load fluctuate in real time. This static parallel configuration cannot intelligently adjust based on real-time performance feedback: when link and system resources are plentiful, a fixed number of concurrent streams cannot fully utilize resources to achieve peak rates; and when network congestion or I / O contention occurs at endpoints, this strategy cannot proactively reduce the parallel scale. Continuous concurrent requests will instead exacerbate resource contention, leading to a surge in retransmission rates and causing the overall migration rate to decrease rather than increase, severely impacting the efficiency and stability of the migration task. Summary of the Invention

[0005] This application provides an adaptive parallel data transmission method and system, which addresses the problem that existing fixed concurrent transmission schemes cannot adapt to environmental changes.

[0006] Firstly, this application provides an adaptive parallel data transmission method, comprising: receiving a transmission request sent by a user and parsing it to obtain data to be transmitted; dividing the data to be transmitted into multiple sub-data blocks; and summarizing the multiple sub-data blocks into a global task list; receiving real-time performance indicator information returned by each server in the server cluster based on a broadcast probe request; analyzing the real-time performance indicator information based on a preset health threshold, and selecting a set of available servers in an idle state from the server cluster based on the real-time performance indicator information; selecting an initial number of servers from the set of available servers as target servers, and obtaining the same number of target sub-data blocks from the global task list, using the initial number as the current parallelism to establish a parallel transmission channel, and transmitting the target sub-data blocks one-to-one to the target servers; calculating the cluster size within the current time window. The current aggregated transmission rate is compared with the historical aggregated transmission rate, which is the aggregated transmission rate within the previous time window. If the current aggregated transmission rate is greater than or equal to the historical aggregated transmission rate, the current parallelism is increased, a new target server is added from the available server set, and the sub-data blocks to be transmitted in the global task list are assigned to the new server. If the current aggregated transmission rate is less than the historical aggregated transmission rate, the current parallelism is decreased, a server is released from the current target server, and the sub-data blocks that were not successfully transmitted in the released server are returned to the global task list. If all sub-data blocks in the global task list have been successfully transmitted, the received target sub-data blocks are retrieved from each target server, and each target sub-data block is concatenated and verified. After verification, the current data transmission ends.

[0007] By adopting the above technical solution, the data to be transmitted is rationally divided and a global task list is established. The performance status of the server cluster is monitored in real time through broadcast probing and health assessment, ensuring the selection of the optimal transmission node from the available server set. During transmission, the aggregated transmission rate is continuously monitored through a time window mechanism, and the current aggregated transmission rate is compared with the historical aggregated transmission rate. When performance improvement is detected, parallelism is increased and new server resources are allocated; when performance degradation is detected, the parallel scale is proactively reduced and unfinished transmission tasks are processed. This adaptive adjustment based on real-time feedback allows the system to maximize transmission performance when resources are abundant and avoid resource constraints in a timely manner, effectively preventing performance crashes caused by excessive resource contention. Simultaneously, the splicing and verification of sub-data blocks ensures transmission reliability, ensuring fast and reliable massive data transmission in complex and ever-changing network environments, significantly improving data migration efficiency.

[0008] Optionally, the aggregate transmission rate within the current time window is calculated, specifically including: receiving the sub-data block transmission completion confirmation information returned by the target server; obtaining the unique identifier of the target sub-data block from the transmission completion confirmation information; storing the unique identifier of the target sub-data block in the data block identifier list corresponding to the current time window; when the current time window ends, retrieving the actual byte size of each identifier from the data block identifier list; accumulating the actual byte sizes corresponding to each identifier to obtain the total amount of data successfully transmitted within the current time window; dividing the total amount of data by the preset duration of the current time window to obtain the current aggregate transmission rate.

[0009] By adopting the above technical solution, a unique identifier is established based on the transmission completion confirmation information and recorded in the identifier list of the current time window. At the end of the time window, the total effective data transmission volume within the time period is accurately calculated by summarizing the actual transmitted byte size of each data block. Combined with the preset time window length, the current aggregated transmission rate is obtained. The rate calculation method using the time window can not only reflect the real-time transmission performance of the system in a timely manner, but also balance the real-time performance and stability of the performance statistics by adjusting the size of the time window.

[0010] Optionally, the current parallelism is increased, and new target servers are added from the available server set. Specifically, this includes: dividing the current aggregated transmission rate by the historical aggregated transmission rate to obtain the performance gain rate; subtracting the performance gain rate from the base to obtain the net gain rate; when the net gain rate is greater than a preset growth trigger threshold, combining multiple historical aggregated transmission rates to obtain a rate sequence, and dividing the standard deviation of the rate sequence by the average value to obtain the performance fluctuation value; obtaining the corresponding average packet loss rate and average latency jitter from multiple target servers, and performing a weighted sum of each average packet loss rate and average latency jitter to obtain the comprehensive network loss value; calculating the performance fluctuation value based on a preset first inverse proportional mapping function to obtain the performance stability index; calculating the comprehensive network loss value based on a preset second inverse proportional mapping function to obtain the network health index; and combining the base growth coefficient and performance... The stability index and network health index are multiplied to obtain the dynamic growth sensitivity coefficient for the current time window. The net gain ratio, current parallelism, and dynamic growth sensitivity coefficient are multiplied to obtain the theoretical increase. The larger value between the theoretical increase and the preset minimum increase step size is selected as the corrected increase. The number of currently unoccupied available idle servers is counted from the available server set. The number of sub-data blocks that have not yet been allocated for transmission is counted from the global task list. The corrected increase, the number of available idle servers, and the number of sub-data blocks are compared, and the minimum value among these factors is selected as the final increase. The same number of servers as the final increase are selected from the available server set as the new target servers. The current parallelism is added to the final increase to update the new current parallelism.

[0011] By adopting the above technical solution, the growth trigger threshold ensures that the system only initiates the expansion process when it has substantial potential for improvement. After determining that expansion conditions are met, the system's performance stability is evaluated by analyzing the dispersion of historical rate sequences. Simultaneously, network quality is quantified by combining the target server's packet loss rate and latency jitter. Performance fluctuation values ​​and network loss values ​​are normalized into stability and health indices using an inverse proportional mapping function. These indicators are then combined with basic growth parameters to obtain a dynamic growth sensitivity coefficient. By comprehensively considering theoretical calculations, resource constraints, and actual limitations such as workload, the most suitable growth scale is selected, effectively preventing resource waste and system instability caused by blind expansion. This ensures that the parallel transmission system always operates at the optimal scale level, ultimately achieving a dynamic balance between transmission efficiency and system stability.

[0012] Optionally, the current parallelism is reduced, and servers are released from the current target servers. Specifically, this includes: dividing the current aggregated transmission rate by the historical aggregated transmission rate to obtain a performance ratio; subtracting the base from the performance ratio to obtain a performance degradation rate; calculating the performance fluctuation value based on a preset first proportional mapping function to obtain a system instability index; calculating the comprehensive network loss value based on a preset second proportional mapping function to obtain a network congestion index; weighted summing the basic reduction coefficient, system instability index, and network congestion index to obtain a dynamic reduction sensitivity coefficient for the current time window; multiplying the performance degradation rate, current parallelism, and dynamic reduction sensitivity coefficient to obtain a theoretical reduction amount; and selecting the larger value between the theoretical reduction amount and a preset minimum reduction step size. The process involves: obtaining the corrected reduction amount; multiplying the current parallelism by the preset maximum reduction ratio to obtain the maximum allowable reduction amount; selecting the minimum value between the corrected reduction amount and the maximum allowable reduction amount to obtain the initial reduction amount; subtracting 1 from the current parallelism to obtain the maximum possible reduction amount; selecting the smaller value between the initial reduction amount and the maximum possible reduction amount as the final reduction amount; obtaining the health score of each target server and sorting the multiple target servers in ascending order based on the health score; selecting the same number of servers as the final reduction amount from the head of the sorting results as servers to be released; releasing the servers to be released and returning the unsuccessfully transmitted sub-data blocks undertaken by each server to the global task list; and subtracting the final reduction amount from the current parallelism to update the new current parallelism.

[0013] By employing the above technical solutions, the system accurately identifies performance degradation trends based on performance ratio and performance degradation rate calculations. Furthermore, the degree of transmission environment degradation is quantitatively assessed using system instability and network congestion indices. These indicators are weighted and combined to form a dynamic reduction sensitivity coefficient, enabling the system to adjust the aggressiveness of its contraction strategy in a timely manner according to the severity of performance degradation and environmental deterioration. When determining the specific contraction scale, a minimum reduction step size ensures the effectiveness of the contraction action. Simultaneously, multiple constraint mechanisms, such as the maximum reduction ratio and maintaining minimum parallelism, prevent performance precipitates caused by excessive contraction. This allows for smooth contraction at the parallel scale when system performance degrades, effectively avoiding the escalation of performance degradation and ensuring the stable operation of the transmission system.

[0014] Optionally, the data to be transmitted is divided into multiple sub-data blocks, and these sub-data blocks are then aggregated into a global task list. Specifically, this involves: starting from the beginning of the data to be transmitted, sliding a window forward on the data in bytes; calculating the rolling hash value using a rolling hash algorithm within the current sliding window; determining whether the rolling hash value meets the splitting condition or whether the number of bytes scanned within the current sliding window has reached the maximum block threshold; when the rolling hash value meets the splitting condition or the number of bytes scanned within the current sliding window has reached the maximum block threshold, marking the end position of the current sliding window as the splitting point; and based on the marked splitting point, dividing the data to be transmitted into multiple sub-data blocks, assigning each sub-data block its own corresponding identifier, and aggregating these into the global task list. The identifier includes a sequence identifier and a checksum for each sub-data block.

[0015] By adopting the above technical solution, a byte-level sliding window mechanism is used to perform fine-grained scanning of the data stream, and a rolling hash algorithm is used to identify natural boundary points at the data content level. This data feature-based segmentation method can dynamically adjust the block size while maintaining the semantic integrity of the data. Furthermore, introducing a maximum block threshold as a mandatory segmentation condition effectively prevents excessively large data blocks caused by uneven distribution of data features, ensuring that the size of each sub-data block remains within a reasonable range, facilitating subsequent parallel transmission and load balancing.

[0016] Optionally, each target sub-data block is concatenated and verified. The data transmission ends after verification. Specifically, this includes: after all sub-data blocks in the global task list are confirmed to have been successfully transmitted, a retrieval command is sent to all participating target servers to obtain all target sub-data blocks and their associated identifiers; based on the identifiers, the pre-stored checksums of individual sub-data blocks are retrieved from the global task list; a hash calculation is performed on the received target sub-data blocks to obtain a real-time checksum; if the checksum of an individual sub-data block does not match the real-time checksum, the target sub-data block is confirmed to be corrupted, and a retransmission request for the target sub-data block is triggered based on the identifier. The process continues until the verification passes. After all retrieved target sub-data blocks pass the block verification, the target sub-data blocks are sorted in ascending order according to the sequence identifier. Based on the sorted order, the byte streams of each sub-data block are concatenated end-to-end to obtain the reconstructed data. A hash calculation is performed on the reconstructed data to obtain the reconstructed data checksum, and a hash calculation is performed on the data to be transmitted to obtain the overall data checksum. If the reconstructed data checksum matches the overall data checksum, the data transmission and concatenation are confirmed to be error-free, and the data transmission ends. If the reconstructed data checksum does not match the overall data checksum, the data transmission or concatenation is confirmed to have failed, and an exception warning is triggered based on the failure.

[0017] By adopting the above technical solution, after confirming the completion of all sub-data block transmissions, data blocks are collected from distributed target servers through a unified retrieval mechanism, and the integrity of the data blocks is ensured through an identifier association mechanism. Before data reassembly, the integrity of each sub-data block is independently verified by comparing the pre-stored checksum with the real-time calculated checksum. When data corruption is detected, a retransmission mechanism can be triggered in a timely manner. This fine-grained verification mechanism effectively prevents overall failure caused by corrupted data blocks participating in the reassembly. After completing the block verification, the correct ordering of data blocks is ensured using sequence identifiers, and the original data is reconstructed through precise splicing at the byte stream level. The reassembly strategy based on sequence identifiers ensures both data integrity and maintains byte-level accuracy.

[0018] Optionally, based on a preset health threshold, real-time performance metrics are analyzed to filter out a set of available servers in an idle state from the server cluster. Specifically, this includes: obtaining load rate, memory utilization, network egress bandwidth utilization, and network latency from the real-time performance metrics; obtaining the overall data volume and data type of the data to be transmitted, and applying a set of weighting coefficients based on the overall data volume and data type; assigning these weighting coefficients sequentially to the load rate, memory utilization, network egress bandwidth utilization, and network latency; multiplying the load rate, memory utilization, network egress bandwidth utilization, and network latency by their respective weighting coefficients for each server in the server cluster, and summing all products to obtain a comprehensive health score for each server; confirming that the server is in an idle state when the comprehensive health score is greater than or equal to the preset health threshold; confirming that the server is in a non-idle state when the comprehensive health score is less than the preset health threshold; and aggregating the idle servers in the server cluster to obtain a set of available servers.

[0019] By adopting the above technical solution, key performance parameters, including load rate, memory utilization, network bandwidth utilization, and network latency, are extracted from real-time performance indicators. By analyzing the scale and type characteristics of the data to be transmitted, the weighting coefficients of each indicator are dynamically adjusted. The weighting allocation strategy based on the characteristics of the transmission task allows the health assessment to better adapt to the resource requirements of different types of data transmission. A weighted summation method is used to integrate multi-dimensional indicators into a single comprehensive health score, preserving the influence of each indicator while providing a unified evaluation standard. By comparing with a preset health threshold, the idle status of the server can be accurately determined, and ultimately, a set of truly usable servers can be selected.

[0020] The second aspect of this application provides an adaptive parallel data transmission system. The system includes a receiving unit, a processing unit, and a verification unit. The receiving unit receives a transmission request sent by a user and parses it to obtain the data to be transmitted. It divides the data into multiple sub-data blocks and summarizes these sub-data blocks into a global task list. It also receives real-time performance indicator information returned by each server in the server cluster based on a broadcast probe request. The processing unit analyzes the real-time performance indicator information based on a preset health threshold, filters out a set of available servers in an idle state from the server cluster, selects an initial number of servers from the available server set as target servers, obtains the same number of target sub-data blocks from the global task list, uses the initial number as the current parallelism to establish a parallel transmission channel, and transmits the target sub-data blocks one-to-one to the target servers. The system calculates the aggregated transmission rate within the current time window and compares it with the historical aggregated transmission rate, which is the aggregated transmission rate within the previous time window. If the current aggregated transmission rate is greater than or equal to the historical aggregated transmission rate, the current parallelism is increased, and a new target server is added from the available server set. The sub-data blocks to be transmitted in the global task list are assigned to the new server. If the current aggregated transmission rate is less than the historical aggregated transmission rate, the current parallelism is decreased, and the server is released from the current target server. The untransmitted sub-data blocks from the released server are returned to the global task list. A verification unit checks the received target sub-data blocks from each target server after all sub-data blocks in the global task list have been successfully transmitted. The unit then concatenates and verifies the target sub-data blocks, ending the data transmission if the verification is successful.

[0021] In a third aspect, this application provides an electronic device including a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory, causing the electronic device to perform the method as described in any of the above-described methods of this application.

[0022] In a fourth aspect, this application provides a computer-readable storage medium storing instructions that, when executed, perform any of the methods described above in this application.

[0023] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: 1. The system rationally divides the data to be transmitted and establishes a global task list. Through broadcast probing and health assessment, it monitors the performance status of the server cluster in real time, ensuring the selection of the optimal transmission node from the available server set. During transmission, a time window mechanism continuously monitors the aggregated transmission rate and compares the current aggregated transmission rate with historical aggregated transmission rates. When performance improvement is detected, parallelism is increased and new server resources are allocated; when performance degradation is detected, the parallel scale is proactively reduced and unfinished transmission tasks are processed. This adaptive adjustment based on real-time feedback allows the system to maximize transmission performance when resources are abundant and avoid resource constraints in a timely manner, effectively preventing performance crashes caused by excessive resource contention. Simultaneously, the splicing and verification of sub-data blocks ensures transmission reliability, guaranteeing fast and reliable massive data transmission in complex and ever-changing network environments, significantly improving data migration efficiency. Attached Figure Description

[0024] Figure 1 This is a flowchart illustrating an adaptive parallel data transmission method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the structure of an adaptive parallel data transmission system provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application.

[0025] Explanation of reference numerals in the attached drawings: 201, receiving unit; 202, processing unit; 203, verification unit; 300, electronic device; 301, processor; 302, memory; 303, user interface; 304, network interface; 305, communication bus. Detailed Implementation

[0026] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0027] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0028] In the description of the embodiments of this application, the term "multiple" means two or more. For example, multiple systems means two or more systems, and multiple screen terminals means two or more screen terminals. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined with "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0029] Therefore, how to adapt to environmental changes when existing fixed-concurrency transmission schemes fail is a pressing problem that needs to be solved. This application provides an adaptive parallel data transmission method, applied to a transmission system. The transmission system of this application can be a data migration platform. Figure 1 This is a flowchart illustrating an adaptive parallel data transmission method provided in an embodiment of this application. (Refer to...) Figure 1 The method includes the following steps S101-S108.

[0030] S101: Receive the transmission request sent by the user and parse it to obtain the data to be transmitted. Divide the data to be transmitted into multiple sub-data blocks and summarize the multiple sub-data blocks into the global task list.

[0031] In step S101 above, when an enterprise needs to migrate data, relevant personnel send a transmission request to the transmission system through the system interface. The transmission request includes the storage location of the data to be transmitted at the source end, the storage address at the target end, and relevant configuration parameters for data migration. After receiving the transmission request, the transmission system parses the content of the transmission request to determine the specific location and attribute information of the data to be transmitted.

[0032] To achieve efficient parallel data transmission, the data to be transmitted needs to be divided into multiple appropriately sized sub-data blocks. This process involves: starting from the beginning of the data to be transmitted, sliding a window forward on the data in bytes; calculating the rolling hash value using a rolling hash algorithm; determining whether the rolling hash value meets the splitting condition or whether the number of bytes scanned in the current sliding window has reached the maximum block threshold; marking the end of the current sliding window as the splitting point when the rolling hash value meets the splitting condition or the number of bytes scanned in the current sliding window has reached the maximum block threshold; and further dividing the data to be transmitted based on the marked splitting point to obtain multiple sub-data blocks, assigning each sub-data block its own identifier, and adding these identifiers to the global task list. The identifiers include a sequence identifier and a checksum for each sub-data block.

[0033] Specifically, the data is divided into blocks starting from the beginning of the data to be transmitted. Using bytes as the basic unit, a 4KB sliding window is created, moving byte by byte across the data stream from left to right. For example, for a 100MB data file, the window initially resides in bytes 0-4095 and then gradually moves forward. During the window's movement, the Rabin-Karp rolling hash algorithm is used to calculate the hash value within the current window in real time. This algorithm maps the byte sequence within the window to a 32-bit integer value, the rolling hash value, using polynomial calculations. When the window moves forward one byte, the algorithm does not need to recalculate the hash value of the entire window; instead, it quickly updates the hash value by removing the contribution of the leftmost byte and adding the contribution of the newly entered byte. This incremental calculation method significantly improves the efficiency of data scanning. After calculating the rolling hash value, it is then determined whether the rolling hash value meets the preset partitioning conditions. Specifically, if the rolling hash value equals 0 when modulo a prime number (such as 8191), the partitioning conditions are considered met. Simultaneously, it tracks the number of bytes scanned within the current sliding window and compares this number with a preset maximum block threshold (e.g., 16MB). This dual-judgment mechanism ensures that data blocks are neither too small, affecting transmission efficiency, nor too large, affecting parallelism. When a splitting condition is met or the maximum block threshold is reached, the end position of the current sliding window is recorded as a splitting point and marked in the data stream. For example, if the splitting condition is met at byte 5242880, that position is marked as a splitting point. The location information of these splitting points is temporarily stored in an ordered array in memory for subsequent data partitioning.

[0034] Based on the marked split points, the entire data to be transmitted is physically divided. Specifically, the data content between two adjacent split points is read and encapsulated into an independent sub-data block. For each sub-data block, an identifier containing two parts is assigned: an integer value representing the splitting order (incrementing from 0), and a data block checksum calculated using the SHA-256 algorithm. For example, the identifier for the first data block might be {sequence number: 0, checksum: 7e4f...3a2b}. All generated sub-data blocks and their identifiers are organized into a task entry and added to the global task list. The global task list is stored in key-value pairs, where the key is the sequence identifier and the value is a structure containing the data block content, checksum, and other information. This organization allows the system to quickly query and schedule any sub-data block, while the sequence identifier ensures accuracy during subsequent data reassembly, and the checksum ensures reliability of data transmission. The global task list provides a unified task pool for subsequent parallel transmission scheduling and facilitates global monitoring and management of transmission progress, ensuring the reliability of the entire data migration process.

[0035] S102: Receive real-time performance metrics information returned by each server in the server cluster based on broadcast probe requests.

[0036] In S102 above, the 'server' or 'server cluster' in this application embodiment may include physical servers, virtual machines, container instances, or other computing nodes with computing, storage, and networking capabilities. To accurately grasp the real-time status of each server in the server cluster, a broadcast probe mechanism is used to periodically collect server performance information. For example, a UDP broadcast probe request is sent to all servers in the server cluster every second. This request includes specific requirements for performance indicator collection, such as the type of indicator to be collected and the sampling frequency. The broadcast probe request uses a specific data packet format, including fields such as request identifier, timestamp, and collection parameters, ensuring that the server can accurately identify and respond to the probe request. When a server in the server cluster receives a broadcast probe request, each server immediately performs local performance indicator collection. By calling the performance counter interface provided by the operating system, the server collects key performance indicators including CPU load rate, memory usage, network egress bandwidth utilization, and network latency. For example, CPU usage is obtained by reading the / proc / stat file, memory usage is obtained by / proc / meminfo, bandwidth utilization is obtained by network interface statistics, and network latency data is obtained by ping tests. These performance metrics data are encapsulated into a unified format response data packet, which includes the server identifier, collection timestamp, and specific values ​​for each performance metric.

[0037] To ensure the real-time nature and reliability of performance data, each server immediately returns a response packet to the system that initiated the probe via a TCP connection after completing the performance metric collection. The system sets a 200-millisecond response timeout for each probe cycle; if a server fails to return data within the timeout period, it is marked as having an abnormal response. For successfully returned performance data, the system performs preliminary validity verification to ensure the data is within a reasonable range, such as checking if the CPU load rate is between 0-100% and if the memory utilization is a valid percentage.

[0038] S103: Analyze real-time performance metrics based on preset health thresholds, and select a set of available servers in an idle state from the server cluster based on real-time performance metrics.

[0039] In step S103 above, suitable servers need to be selected from the server cluster for data transmission tasks. Real-time performance metrics are analyzed based on a preset health threshold. The set of available servers in an idle state is selected from the server cluster based on the real-time performance metrics. Specifically, this includes: obtaining load rate, memory usage, network egress bandwidth utilization, and network latency from the real-time performance metrics; obtaining the overall data volume and data type of the data to be transmitted, and applying a set of weighting coefficients based on the overall data volume and data type; assigning these weighting coefficients sequentially to the load rate, memory usage, network egress bandwidth utilization, and network latency; multiplying the load rate, memory usage, network egress bandwidth utilization, and network latency of each server in the server cluster by their respective weighting coefficients, and summing all products to obtain a comprehensive health score for each server; confirming that the server is in an idle state when the comprehensive health score is greater than or equal to the preset health threshold; confirming that the server is in a non-idle state when the comprehensive health score is less than the preset health threshold; and summing the idle servers in the server cluster to obtain a set of available servers.

[0040] Specifically, key performance parameters are extracted from the received real-time performance metrics. For each server, CPU load (0-100%), memory utilization (0-100%), network egress bandwidth utilization (0-100%), and network latency (in milliseconds) are read. For example, a server's specific metrics might be: CPU load 30%, memory utilization 45%, bandwidth utilization 25%, and network latency 20ms. This raw data is temporarily stored in a performance metric cache for subsequent health assessments. Next, the characteristics of the data to be transmitted are analyzed, including the overall data volume and data type. Different weighting coefficients are set according to the characteristics of the data to be transmitted. When the data to be transmitted is detected as a large file (such as a video file), the weight of the network bandwidth metric is increased accordingly; when it is identified as small file-intensive data, the weight of the CPU and memory metrics is increased. For example, for a 100GB database backup file, which is identified as a large-scale structured data transfer task, the following weighting might be used for large-scale data transfers: CPU load rate weight 0.2, memory utilization rate weight 0.3, bandwidth utilization rate weight 0.35, and network latency weight 0.15. This task-characteristic-based weighting ensures that the performance evaluation is more in line with actual transmission requirements.

[0041] After determining the weighting coefficients, a comprehensive health assessment is performed on each server. The original performance indicators are normalized, and network latency is converted into a score range of 0-100 (e.g., 50ms or more is recorded as 100 points, and 0ms as 0 points). Linear interpolation is used to calculate the median value. Then, each normalized indicator is multiplied by its corresponding weight, and all weighted scores are summed to obtain the server's comprehensive health score. The calculation formula is: Comprehensive Health Score = (100 - Load Rate) * 0.2 + (100 - Memory Utilization) * 0.3 + (100 - Bandwidth Utilization) * 0.35 + (100 - Latency Score) * 0.15. For the example server above: Comprehensive Health Score = (100 - 30) * 0.2 + (100 - 45) * 0.3 + (100 - 25) * 0.35 + (100 - 40) * 0.15 = 65.75. The calculated overall health score is compared with a preset health threshold, which can be set to 60 points. If the overall health score is greater than or equal to the preset health threshold, the server is marked as idle; otherwise, it is marked as non-idle. A dynamic state mapping table is maintained, recording the state label and latest health score of each server. Finally, the state mapping table is traversed, and all servers marked as idle are selected to form an available server set. This set is stored using a priority queue data structure, sorted in descending order of overall health score, facilitating subsequent selection of the optimal server for task allocation. This available server set is also periodically updated to ensure that the server status in the set continuously meets transmission requirements.

[0042] S104: Select an initial number of servers from the available server set as target servers, and obtain the same number of target sub-data blocks from the global task list. Use the initial number as the current parallelism to establish a parallel transmission channel, and transmit the target sub-data blocks one-to-one to the target server.

[0043] In step S104 above, parallel transmission is determined based on available resources and task characteristics. From the filtered set of available servers, a top-ranked initial number of servers are selected as initial target servers based on their comprehensive health scores; for example, the initial number can be set to 8. The selection of the initial number considers the rational utilization of network bandwidth resources and the balance of scheduling overhead, avoiding resource contention caused by excessive parallelism. Eight unallocated target sub-data blocks are retrieved from the global task list according to their sequential identifiers. For example, if there are 100 sub-data blocks in the global task list, the first 8 sub-data blocks with identifiers 0 to 7 are selected first. Each selected sub-data block is marked as "allocated" in the global task list to avoid duplicate allocation. This sequential block selection strategy helps maintain the orderliness of data transmission and simplifies the subsequent data reassembly process.

[0044] The initial value of 8 is set as the current parallelism parameter, and parallel transmission channels are established. An independent transmission session is created for each pair of servers and target sub-blocks. Each session includes components such as a TCP connection from the source to the target server, block identification information, and a transmission progress tracker. For example, the first transmission session might assign the sub-block with identifier 0 to the target server with the highest health score, establishing a dedicated data transmission channel. A one-to-one mapping strategy is used when establishing transmission channels to ensure that each target server handles only one sub-block transmission task at a time. This one-to-one correspondence is recorded in a transmission mapping table, containing information such as server identifier, block identifier, and transmission start time. For example, a record in the transmission mapping table might be: {Server ID:server001, Block ID:block_0, Start Time:2025-11-04 10:30:00}. Once all transmission channels are established, parallel data transmission begins. Each transmission channel operates independently, ensuring data transmission reliability through the TCP protocol, while monitoring the transmission status and progress of each channel.

[0045] S105: Calculate the aggregated transmission rate within the current time window and compare the current aggregated transmission rate with the historical aggregated transmission rate.

[0046] In step S105 above, the effectiveness of the current parallel transmission strategy is evaluated by monitoring changes in the transmission rate. A fixed-size time window, such as 10 seconds, is first set to collect and statistically analyze transmission performance data. Within each time window, the data transmission volume of all parallel transmission channels is recorded in real time, including the number of bytes transmitted for each sub-data block and the transmission duration. The aggregate transmission rate within the current time window is calculated, specifically including: receiving the sub-data block transmission completion confirmation information returned by the target server; obtaining the unique identifier of the target sub-data block from the transmission completion confirmation information; storing the unique identifier of the target sub-data block in the data block identifier list corresponding to the current time window; when the current time window expires, retrieving the actual byte size of each identifier from the data block identifier list; accumulating the actual byte sizes corresponding to each identifier to obtain the total amount of data successfully transmitted within the current time window; and dividing the total data amount by the preset duration of the current time window to obtain the current aggregate transmission rate.

[0047] Specifically, it is necessary to accurately calculate the actual data transmission efficiency within each time window, creating a dedicated data structure for each time window to record the information of data blocks that have been transmitted within that window. This data structure includes fields such as the window start timestamp, end timestamp, and a list of data block identifiers. When the target server completes the transmission of a sub-data block, it immediately sends a transmission completion confirmation message to the system. This confirmation message is encapsulated in JSON format and contains several key fields. Upon receiving the confirmation message, the system first verifies the integrity and authenticity of the message, and then parses out the unique identifier of the target sub-data block. Based on the completion timestamp in the confirmation message, the identifier of the data block is added to the data block identifier list of the corresponding time window. An active window cache is maintained to store all relevant information for the current active time window. For example, if a data block is transmitted at 10:30:05, and the current active window is 10:30:00-10:30:10, then the identifier of the data block is added to the identifier list of this window. When the time window reaches the preset end time (e.g., 10 seconds later), the transmission performance within that window is calculated. The actual size of each data block in the identifier list is queried from the global task list. For example, a data block identified as "block_123" might be 16MB in size. The system uses an accumulator variable to iterate through all data blocks in the identifier list, accumulating their actual byte size one by one. After obtaining the total data volume, the total data volume is divided by the preset duration of the time window (e.g., 10 seconds) to calculate the aggregate transfer rate for that window. For example, if a total of 4000MB of data is transferred within a 10-second time window, the aggregate transfer rate is 400MB / s.

[0048] Furthermore, the calculated current aggregated transmission rate is compared with historical aggregated transmission rates. If the current time window is the first time window, the current aggregated transmission rate is recorded as a baseline value for comparison in subsequent windows; if the current time window is a subsequent time window, it is compared with the aggregated transmission rate of the previous time window. A rate history table is maintained to record the aggregated transmission rate for each time window, including timestamp, aggregated transmission rate, parallelism, and other information. Time window-based transmission rate monitoring and comparison can accurately evaluate the effectiveness of the current parallel transmission strategy. The windowed data acquisition method not only smooths out the impact of instantaneous fluctuations but also provides a stable performance benchmark.

[0049] S106: If the current aggregated transmission rate is greater than or equal to the historical aggregated transmission rate, the current parallelism is increased, and a new target server is added from the set of available servers. The sub-data blocks to be transmitted in the global task list are assigned to the newly added server.

[0050] In S106 above, if only performance gains are considered when making decisions to increase parallelism, it may lead to over-expansion in an unstable network environment. Therefore, this embodiment comprehensively considers the stability of historical performance and the current health of the network to jointly adjust the sensitivity of growth, thereby achieving the expansion of parallel transmission. The transmission parallelism is dynamically expanded according to the performance comparison results. When the aggregated transmission rate of the current time window is detected to be greater than or equal to the historical aggregated transmission rate, it indicates that there is still room for optimization in the current parallel transmission strategy, triggering the parallelism enhancement mechanism. For example, if the current aggregated transmission rate is 400MB / s and the historical aggregated transmission rate is 380MB / s, then the parallelism enhancement is triggered.

[0051] Furthermore, the current parallelism is increased, and new target servers are added from the available server set. Specifically, this includes: dividing the current aggregated transmission rate by the historical aggregated transmission rate to obtain the performance gain rate; subtracting the performance gain rate from the base to obtain the net gain rate; when the net gain rate is greater than a preset growth trigger threshold, combining multiple historical aggregated transmission rates to obtain a rate sequence, and dividing the standard deviation of the rate sequence by the average value to obtain the performance fluctuation value; obtaining the corresponding average packet loss rate and average latency jitter from multiple target servers, and performing a weighted sum of each average packet loss rate and average latency jitter to obtain the comprehensive network loss value; calculating the performance fluctuation value based on a preset first inverse proportional mapping function to obtain the performance stability index; calculating the comprehensive network loss value based on a preset second inverse proportional mapping function to obtain the network health index; and combining the basic growth coefficient and performance stability index. The qualitative index and the network health index are multiplied to obtain the dynamic growth sensitivity coefficient for the current time window. The net gain ratio, the current parallelism, and the dynamic growth sensitivity coefficient are multiplied to obtain the theoretical increase. The larger value between the theoretical increase and the preset minimum increase step size is selected as the corrected increase. The number of currently unoccupied available idle servers is counted from the available server set. The number of sub-data blocks that have not yet been allocated for transmission is counted from the global task list. The corrected increase, the number of available idle servers, and the number of sub-data blocks are compared, and the minimum value among these factors is selected as the final increase. The same number of servers as the final increase are selected from the available server set as the new target servers. The current parallelism is added to the final increase to update the new current parallelism.

[0052] Specifically, the performance gain is first calculated by dividing the aggregated transmission rate of the current time window by the aggregated transmission rate of the historical time windows to obtain the performance gain ratio. This gain ratio is then subtracted from the base to obtain the net gain ratio, which reflects the degree of performance improvement relative to the previous state. For example, if the current aggregated transmission rate is 400 MB / s and the historical aggregated transmission rate is 380 MB / s, the performance gain ratio is 1.053, and the base is 1. Subtracting the base from the performance gain ratio yields a net gain ratio of 0.053. When the net gain ratio exceeds a preset growth trigger threshold, in-depth performance stability analysis begins, if the preset growth trigger threshold is set to 0. Aggregated transmission rate data for the most recent five time windows are collected, such as [380, 385, 395, 390, 400] MB / s, to construct a rate sequence. The standard deviation (approximately 7.91) and average (390) of this sequence are calculated. Dividing the standard deviation by the average yields a performance fluctuation value of 0.0203. This indicator reflects the stability of the system performance. Simultaneously, network quality data for all current target servers is collected. For example, for the eight target servers, their average packet loss rate [0.1%, 0.2%, 0.15%, 0.12%, 0.18%, 0.14%, 0.16%, 0.13%] and average latency jitter [5ms, 6ms, 4ms, 5ms, 7ms, 5ms, 6ms, 5ms] are calculated. These metrics are assigned weights (packet loss rate weight 0.6, latency jitter weight 0.4), resulting in a comprehensive network loss value of 0.0148.

[0053] The performance stability index is calculated using a preset first inverse proportional mapping function, which is defined as 1 / (1 + adjustable constant * performance fluctuation value). With the adjustable constant set to 10, substituting this into the function 1 / (1 + 10 * 0.0203) yields a performance stability index of 0.831. Similarly, the network health index is calculated using a preset second inverse proportional mapping function, defined as max(0, 1 - adjustable constant * comprehensive network loss value). Substituting this into the function max(0, 1 - 10 * 0.0148) yields a network health index of 0.852. Both the performance stability index and the network health index are between 0 and 1, reflecting the stability of system performance and the health of the network environment, respectively. Multiplying the calculated performance stability index, network health index, and basic growth coefficient yields the dynamic growth sensitivity coefficient for the current time window. For example, if the basic growth coefficient is set to 0.25, the dynamic growth sensitivity coefficient is 0.177. This dynamic growth sensitivity coefficient comprehensively reflects the impact of system performance stability and network environment quality on parallelism growth.

[0054] Next, multiply the net gain ratio, current parallelism, and dynamic growth sensitivity coefficient to obtain the theoretical increase. Compare this theoretical value with the preset minimum increase step size and select the larger value as the corrected increase. For example, with a net gain ratio of 0.053, a current parallelism of 8, and a dynamic growth sensitivity coefficient of 0.177, the theoretical increase = 0.053 * 8 * 0.177 = 0.075. The preset minimum increase step size is set to 1. Compare this theoretical value with the preset minimum increase step size of 1 and select the larger value of 1 as the corrected increase. By querying the resource manager, it is determined that there are 3 idle servers in the current available server set, and a scan of the global task list reveals 5 unallocated sub-data blocks. Compare the corrected increase, the number of available idle servers, and the number of unallocated sub-data blocks, and select the minimum value of 1 as the final increase. Select the server with the highest overall health score from the available server set as the new target server. The calculation of the overall health score considers multiple indicators such as server packet loss rate and latency jitter to ensure the selection of the best server resources. Increase the current parallelism from 8 to 9, and record the detailed information of this adjustment in the parallelism management table, including the adjustment time, the parallelism values ​​before and after the adjustment, and various performance indicators.

[0055] Furthermore, after identifying the new servers, an independent transmission channel is established for each pair of new servers and data blocks to facilitate data transmission. Through this dynamic scaling method, when performance improvement is detected, transmission parallelism is increased in a timely manner to fully utilize network resources. The new transmission channels work in conjunction with existing channels to further improve overall transmission efficiency.

[0056] S107: If the current aggregated transmission rate is less than the historical aggregated transmission rate, reduce the current parallelism and release the server from the current target server, returning the untransmitted sub-data blocks in the released server to the global task list.

[0057] In S107 above, the parallel transmission strategy is adjusted in a timely manner when a performance degradation is detected. When the aggregate transmission rate of the current time window is less than the aggregate transmission rate of the historical time window, it indicates that the current parallel transmission strategy may lead to excessive competition for network resources or excessive server load, and the parallelism reduction mechanism needs to be triggered.

[0058] Furthermore, the current parallelism is reduced, and servers are released from the current target servers. Specifically, this includes: dividing the current aggregated transmission rate by the historical aggregated transmission rate to obtain the performance ratio; subtracting the base from the performance ratio to obtain the performance degradation rate; calculating the performance fluctuation value based on a preset first proportional mapping function to obtain the system instability index; calculating the comprehensive network loss value based on a preset second proportional mapping function to obtain the network congestion index; weighted summing the basic reduction coefficient, the system instability index, and the network congestion index to obtain the dynamic reduction sensitivity coefficient for the current time window; multiplying the performance degradation rate, the current parallelism, and the dynamic reduction sensitivity coefficient to obtain the theoretical reduction amount; and selecting the larger value between the theoretical reduction amount and the preset minimum reduction step size. The process involves: obtaining the corrected reduction amount; multiplying the current parallelism by the preset maximum reduction ratio to obtain the maximum allowable reduction amount; selecting the minimum value between the corrected reduction amount and the maximum allowable reduction amount to obtain the initial reduction amount; subtracting 1 from the current parallelism to obtain the maximum possible reduction amount; selecting the smaller value between the initial reduction amount and the maximum possible reduction amount as the final reduction amount; obtaining the health score of each target server and sorting the multiple target servers in ascending order based on their health scores; selecting the same number of servers as the final reduction amount from the top of the sorting results as servers to be released; releasing the servers to be released and returning the unsuccessfully transmitted sub-data blocks undertaken by each server to the global task list; and subtracting the final reduction amount from the current parallelism to update the current parallelism.

[0059] Specifically, dividing the current aggregated transfer rate by the historical aggregated transfer rate yields the performance ratio, indicating a declining trend in system performance. Subtracting the performance ratio from the base yields the performance degradation rate, which quantifies the degree of performance decline. For example, with a current aggregated transfer rate of 380 MB / s, a historical aggregated transfer rate of 400 MB / s, a performance ratio of 0.95, and a base of 1, subtracting the performance ratio of 0.95 from the base of 1 yields a performance degradation rate of 0.05. Extracting the aggregated transfer rates [400, 395, 385, 382, ​​380] for the most recent five consecutive time windows from the historical performance database, the performance fluctuation value is calculated to be 0.0203. Meanwhile, real-time network quality data was obtained from the performance monitoring modules of each target server. The average packet loss rates of the 10 target servers were [0.2%, 0.15%, 0.25%, 0.18%, 0.3%, 0.22%, 0.28%, 0.16%, 0.32%, 0.24%], and the average latency jitter were [8ms, 6ms, 9ms, 7ms, 11ms, 8ms, 10ms, 6ms, 12ms, 9ms]. These metrics were weighted (packet loss rate weighted at 0.6, latency jitter weighted at 0.4) to obtain a comprehensive network loss value of 0.0246.

[0060] Next, the performance fluctuation value is processed using a preset first proportional mapping function, min(1, adjustable constant * performance fluctuation value). The adjustable constant is set to 5, and the performance fluctuation value of 0.0203 is substituted into the function to calculate the system instability index, which is 0.1015. Then, the comprehensive network loss value is processed using a preset second proportional mapping function, min(1, adjustable constant * comprehensive network loss value). The comprehensive network loss value of 0.0246 is substituted into the function to obtain the network congestion index, which is 0.123. The basic reduction coefficient of 0.3, the system instability index of 0.1015, and the network congestion index of 0.123 are weighted and summed according to weights [0.4, 0.3, 0.3] to obtain the dynamic reduction sensitivity coefficient of 0.174. The dynamic reduction sensitivity coefficient comprehensively reflects the sensitivity of the system state to parallelism reduction. Multiplying the performance degradation rate of 0.05, the current parallelism of 10, and the dynamic reduction sensitivity coefficient of 0.174 yields a theoretical reduction of 0.087. This theoretical reduction is compared to the preset minimum reduction step size of 1, and the larger value of 1 is selected as the corrected reduction amount. The preset maximum reduction ratio is set to 0.3, and the current parallelism of 10 is multiplied by 0.3 to obtain the maximum allowable reduction amount of 3.

[0061] Furthermore, comparing the reduced quantity of 1 and the maximum allowed reduced quantity of 3, the smaller value of 1 is selected as the initial reduced quantity. Subtracting 1 from the current parallelism of 10 yields the maximum allowable reduced quantity of 9. Comparing the initial reduced quantity of 1 with the maximum allowable reduced quantity of 9, the smaller value of 1 is selected as the final reduced quantity. The health score of each target server is calculated. The health score comprehensively considers indicators such as server CPU utilization, memory usage, network latency, and packet loss rate. The specific calculation formula is: 10 - CPU utilization 0.3 - Memory usage 0.2 - Standardized network latency 0.25 - Packet loss rate 100 * 0.25. The resulting health scores for the 10 servers are [85, 88, 82, 87, 75, 84, 80, 89, 73, 86]. These servers are sorted in ascending order of health score, resulting in the following sorted list: [73 (server9), 75 (server5), 80 (server7), 82 (server3), 84 (server6), 85 (server1), 86 (server10), 87 (server4), 88 (server2), 89 (server8)]. One server (server9) is selected from the top of the sorted list (finally reduced in number) as the server to be released. The transmission task status on server9 is checked, and the completion rate of the sub-data block block15 being transmitted is found to be 45%. This incomplete sub-data block is returned to the global task list, and its task status is updated to "pending transmission." The transmission connection with server9 is closed, relevant session information and cached data are cleared, its status is updated to "idle," and it is added to the available server set. The current parallelism is reduced from 10 to 9, and this adjustment is recorded in the parallelism management table.

[0062] The dynamic shrinkage mechanism described above can reduce the scale of parallel transmissions in a timely manner when performance degradation is detected, free up server resources with poor performance, and ensure that unfinished transmission tasks can be reasonably saved and reallocated.

[0063] S108: If all sub-data blocks in the global task list have been successfully transmitted, retrieve the received target sub-data blocks from each target server, and concatenate and verify each target sub-data block. If the verification is successful, end the data transmission.

[0064] In S108 above, by precisely controlling the parallelism within each time window, the transmission system is ensured to always operate at the optimal parallel scale. The transmission status of all sub-data blocks in the global task list is checked. When the status of the last sub-data block in the global task list also changes to "transmission complete," it is confirmed that all sub-data blocks have been successfully transmitted, and the data integration process is then initiated.

[0065] Each target sub-data block is concatenated and verified. Once verification is successful, the data transmission ends. Specifically, this includes: after all sub-data blocks in the global task list have been confirmed as successfully transmitted, a retrieval command is sent to all participating target servers to obtain all target sub-data blocks and their associated identifiers; based on the identifiers, the pre-stored checksums of individual sub-data blocks are retrieved from the global task list; a hash calculation is performed on the received target sub-data blocks to obtain a real-time checksum; if the checksum of an individual sub-data block does not match the real-time checksum, the target sub-data block is confirmed to be corrupted, and a retransmission request for the target sub-data block is triggered based on the identifier, until the checksum is corrected. The verification is passed; after all the retrieved target sub-data blocks have passed the block verification, the target sub-data blocks are sorted in ascending order according to the sequence identifier; based on the sorted order, the byte streams of each sub-data block are concatenated end-to-end to obtain the reconstructed data; a hash calculation is performed on the reconstructed data to obtain the reconstructed data checksum, and a hash calculation is performed on the data to be transmitted to obtain the overall data checksum; if the reconstructed data checksum is consistent with the overall data checksum, the data transmission and concatenation are confirmed to be error-free, and the data transmission ends; if the reconstructed data checksum is inconsistent with the overall data checksum, the data transmission or concatenation is confirmed to have failed, and an exception warning is triggered based on the failure.

[0066] Specifically, once the transmission status of all sub-data blocks in the global task list is detected as successfully transmitted, the data aggregation process is initiated, simultaneously sending retrieval commands to all participating target servers (assuming 10 servers). Upon receiving the command, each server packages and returns its stored target sub-data blocks and their associated identifiers. Then, single-block data verification is performed. For each received target sub-data block, the pre-stored checksum is retrieved from the global task list based on its identifier (e.g., "block_001"). The received target sub-data block is calculated using the SHA-256 hash algorithm to obtain a real-time checksum. When the real-time checksum "7a9c4b2e1f8d3a" of a sub-data block (e.g., block_045) is found to be inconsistent with the pre-stored checksum "8f7d88e4c2a23c", the block is immediately marked as corrupted. A retransmission request is generated for the marked corrupted sub-data block, added to the transmission queue, and the retransmission process is continuously monitored until verification passes. After retransmission, the status information of the sub-data block is updated. After all sub-data blocks pass single-block verification, they are sorted based on their sequence identifiers. A sorted array is constructed, with block_001 to block_100 sorted in ascending order by sequence_id to ensure the correct order of the data blocks. The data reconstruction process then begins. A dynamically growing byte buffer is created, and the binary data of each sub-data block is written to the buffer sequentially according to the sorted order. Write pointers are maintained during the writing process to ensure seamless concatenation between data blocks.

[0067] After data concatenation is complete, a SHA-256 hash is performed on the reconstructed complete data to obtain the reconstructed data checksum "9e8d7c6b5a4f3e2d1". Simultaneously, the overall checksum of the original data to be transmitted, also "9e8d7c6b5a4f3e2d1", is retrieved from the task configuration and compared for verification. If the reconstructed data checksum and the overall data checksum are found to be a perfect match, a transmission success report is generated. Subsequently, temporary storage space is cleared, related resources are released, and the transmission task is marked as complete. If an inconsistency is detected between the reconstructed data checksum (e.g., "8a7b6c5d4e3f2g1h") and the overall data checksum ("9e8d7c6b5a4f3e2d1"), an anomaly warning mechanism is immediately triggered, generating a detailed error report, which is then sent to the monitoring system and the administrator.

[0068] This multi-layered verification and reconstruction mechanism ensures the integrity and accuracy of distributed data transmission. Single-block verification ensures the correctness of each sub-data block, sequential concatenation guarantees the accuracy of data reconstruction, and overall verification provides final data consistency assurance. Simultaneously, the anomaly handling mechanism promptly detects and reports problems, ensuring the reliability of data transmission. This solution not only effectively prevents errors and corruption during data transmission but also provides automatic repair through retransmission mechanisms, offering a complete data quality assurance system.

[0069] This application also provides an adaptive parallel data transmission system. Figure 2 This is a schematic diagram of an adaptive parallel data transmission system provided in an embodiment of this application. (Refer to...) Figure 2 The system includes a receiving unit 201, a processing unit 202, and a verification unit 203; The receiving unit 201 receives the transmission request sent by the user and parses it to obtain the data to be transmitted. It divides the data to be transmitted into multiple sub-data blocks and summarizes the multiple sub-data blocks into a global task list. It also receives real-time performance indicator information returned by each server in the server cluster based on the broadcast probe request. Processing unit 202 analyzes real-time performance metrics based on a preset health threshold, filters out available servers in the idle state from the server cluster, selects an initial number of servers from the available server set as target servers, and obtains the same number of target sub-data blocks from the global task list. The initial number is used as the current parallelism to establish a parallel transmission channel, and the target sub-data blocks are transmitted one-to-one to the target servers. The aggregate transmission rate within the current time window is calculated and compared with the historical aggregate transmission rate, which is the aggregate transmission rate within the previous time window. If the current aggregate transmission rate is greater than or equal to the historical aggregate transmission rate, the current parallelism is increased, and a new target server is added from the available server set. The sub-data blocks to be transmitted in the global task list are allocated to the new server. If the current aggregate transmission rate is less than the historical aggregate transmission rate, the current parallelism is decreased, and the server is released from the current target server. The untransmitted sub-data blocks from the released server are returned to the global task list. Verification unit 203, if all sub-data blocks in the global task list have been successfully transmitted, retrieves the received target sub-data blocks from each target server, splices and verifies each target sub-data block, and ends the data transmission after verification is successful.

[0070] In one possible implementation, the receiving unit 201 is used to receive the sub-data block transmission completion confirmation information returned by the target server, obtain the unique identifier of the target sub-data block from the transmission completion confirmation information, and store the unique identifier of the target sub-data block in the data block identifier list corresponding to the current time window; the processing unit 202 is used to retrieve the actual byte size of each identifier from the data block identifier list when the current time window ends, accumulate the actual byte sizes corresponding to each identifier to obtain the total amount of data successfully transmitted within the current time window; and divide the total amount of data by the preset duration of the current time window to obtain the current aggregate transmission rate.

[0071] In one possible implementation, the processing unit 202 is configured to divide the current aggregated transmission rate by the historical aggregated transmission rate to obtain the performance gain rate; subtract the performance gain rate from the base to obtain the net gain rate; when the net gain rate is greater than a preset growth trigger threshold, combine multiple historical aggregated transmission rates to obtain a rate sequence; divide the standard deviation of the rate sequence by the average value to obtain the performance fluctuation value; obtain the corresponding average packet loss rate and average latency jitter from multiple target servers, and perform a weighted summation of each average packet loss rate and average latency jitter to obtain a comprehensive network loss value; calculate the performance fluctuation value based on a preset first inverse proportional mapping function to obtain a performance stability index; calculate the comprehensive network loss value based on a preset second inverse proportional mapping function to obtain a network health index; and combine the basic growth coefficient, performance stability index, and network health index... Multiply the Kang index to obtain the dynamic growth sensitivity coefficient for the current time window; multiply the net gain ratio, current parallelism, and dynamic growth sensitivity coefficient to obtain the theoretical increase; select the larger value between the theoretical increase and the preset minimum increase step size as the corrected increase; count the number of currently unoccupied available idle servers from the available server set; count the number of currently unallocated sub-data blocks from the global task list; compare the corrected increase, the number of available idle servers, and the number of sub-data blocks, and select the minimum value as the final increase; select the same number of servers as the final increase from the available server set as the new target servers; add the final increase to the current parallelism to update the new current parallelism.

[0072] In one possible implementation, the processing unit 202 is used to divide the current aggregated transmission rate by the historical aggregated transmission rate to obtain a performance ratio; subtract the base from the performance ratio to obtain a performance degradation rate; calculate the performance fluctuation value based on a preset first proportional mapping function to obtain a system instability index; calculate the comprehensive network loss value based on a preset second proportional mapping function to obtain a network congestion index; perform a weighted summation of the basic reduction coefficient, the system instability index, and the network congestion index to obtain a dynamic reduction sensitivity coefficient for the current time window; multiply the performance degradation rate, the current parallelism, and the dynamic reduction sensitivity coefficient to obtain a theoretical reduction amount; select the larger value between the theoretical reduction amount and the preset minimum reduction step size to obtain a corrected reduction amount; and multiply the current parallelism by... The maximum allowable reduction quantity is obtained by using a preset maximum reduction ratio; the minimum value between the corrected reduction quantity and the maximum allowable reduction quantity is selected to obtain the initial reduction quantity; the current parallelism is subtracted by 1 to obtain the maximum possible reduction quantity; the smaller value between the initial reduction quantity and the maximum possible reduction quantity is selected as the final reduction quantity; the receiving unit 201 is used to obtain the health score of each target server and sort the multiple target servers in ascending order according to the health score; the processing unit 202 is used to select the same number of servers as the final reduction quantity from the head of the sorting result as servers to be released, release the servers to be released, and return the unsuccessfully transmitted sub-data blocks undertaken by each server to the global task list; the current parallelism is subtracted from the final reduction quantity to update the new current parallelism.

[0073] In one possible implementation, the processing unit 202 is configured to slide a window forward on the data to be transmitted, starting from the beginning position of the data to be transmitted, in bytes; calculate the rolling hash value of the data in the current sliding window using a rolling hash algorithm; determine whether the rolling hash value meets the splitting condition, or whether the number of bytes scanned in the current sliding window has reached the maximum block threshold; when the rolling hash value meets the splitting condition, or the number of bytes scanned in the current sliding window has reached the maximum block threshold, confirm that the end position of the current sliding window is marked as the splitting point; divide the data to be transmitted based on the marked splitting point to obtain multiple sub-data blocks, assign each sub-data block its own corresponding identifier, and summarize them into a global task list, wherein the identifier includes a sequence identifier and a checksum of a single sub-data block.

[0074] In one possible implementation, processing unit 202 is used to, after all sub-data blocks in the global task list have been confirmed to have been successfully transmitted, send a retrieval command to all target servers participating in the transmission, obtain all target sub-data blocks and their associated identifiers according to the retrieval command; query the pre-stored checksum of a single sub-data block from the global task list based on the identifier, perform hash calculation on the received target sub-data blocks, and obtain a real-time checksum; verification unit 203 is used to, when the checksum of a single sub-data block is inconsistent with the real-time checksum, confirm that the target sub-data block is corrupted, trigger a retransmission request for the target sub-data block based on the identifier, until the verification passes; processing unit 202 uses After all retrieved target sub-data blocks have passed the block verification, the target sub-data blocks are sorted in ascending order according to the sequence identifier. Based on the sorted order, the byte streams of each sub-data block are concatenated end-to-end to obtain the reconstructed data. The reconstructed data is hashed to obtain the reconstructed data checksum, and the data to be transmitted is hashed to obtain the overall data checksum. The verification unit 203 is used to confirm that the data transmission and concatenation are correct if the reconstructed data checksum is consistent with the overall data checksum, and to end the current data transmission. If the reconstructed data checksum is inconsistent with the overall data checksum, the data transmission or concatenation is confirmed to have failed, and an abnormal warning is triggered based on the failure.

[0075] In one possible implementation, the receiving unit 201 is used to obtain load rate, memory utilization rate, network egress bandwidth utilization rate, and network latency from real-time performance indicator information; obtain the overall data volume and data type of the data to be transmitted, and adopt a set of weighting coefficients according to the overall data volume and data type; and assign the set of weighting coefficients to the load rate, memory utilization rate, network egress bandwidth utilization rate, and network latency in sequence; the processing unit 202 is used to multiply the load rate, memory utilization rate, network egress bandwidth utilization rate, and network latency of each server in the server cluster by their respective weighting coefficients, and sum all the products to obtain the comprehensive health score of each server; when the comprehensive health score is greater than or equal to a preset health threshold, the server is confirmed to be in an idle state; when the comprehensive health score is less than the preset health threshold, the server is confirmed to be in a non-idle state, and the idle servers in the server cluster are aggregated to obtain a set of available servers.

[0076] It should be noted that the system provided in the above embodiments is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0077] This application also discloses an electronic device. (See reference...) Figure 3 , Figure 3 This application provides a schematic diagram of the structure of an electronic device. The electronic device 300 may include: at least one processor 301, at least one network interface 304, a user interface 303, a memory 302, and at least one communication bus 305.

[0078] The communication bus 305 is used to enable communication between these components.

[0079] The user interface 303 may include a display screen and a camera. Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0080] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0081] The processor 301 may include one or more processing cores. The processor 301 connects to various parts of the server using various interfaces and lines, and performs various server functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 302, and by calling data stored in memory 302. Optionally, the processor 301 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 301 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and application requests; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 301 and may be implemented as a separate chip.

[0082] The memory 302 may include random access memory (RAM) or read-only memory. Optionally, the memory 302 may include a non-transitory computer-readable storage medium. The memory 302 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 302 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), instructions for implementing the various method embodiments described above, etc. The data storage area may store data involved in the various method embodiments described above. Optionally, the memory 302 may also be at least one storage device located remotely from the aforementioned processor 301.

[0083] like Figure 3 As shown, the memory 302, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program for adaptive parallel data transmission.

[0084] exist Figure 3 In the electronic device 300 shown, the user interface 303 is mainly used to provide an interface for users to input data and obtain user input data; while the processor 301 can be used to call the application program stored in the memory 302 for adaptive parallel data transmission. When executed by one or more processors, the electronic device performs one or more of the methods described in the above embodiments.

[0085] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0086] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0087] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some service interfaces; indirect couplings or communication connections between devices or units may be electrical or other forms.

[0088] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0090] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0091] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of other embodiments of this disclosure upon considering the specification and the disclosure of practical truths. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure.

Claims

1. An adaptive parallel data transmission method, characterized in that, The method includes: Receive the transmission request sent by the user and parse the data to be transmitted. Divide the data to be transmitted into multiple sub-data blocks and summarize the multiple sub-data blocks into the global task list. Based on broadcast probe requests, receive real-time performance metrics information returned by each server in the server cluster. The real-time performance indicator information is analyzed based on a preset health threshold, and the set of available servers in the idle state is selected from the server cluster based on the real-time performance indicator information. An initial number of servers are selected from the available server set as target servers, and target sub-data blocks of the same number as the initial number are obtained from the global task list. The initial number is used as the current parallelism to establish a parallel transmission channel, and the target sub-data blocks are transmitted one-to-one to the target server. Calculate the aggregated transmission rate within the current time window, and compare the current aggregated transmission rate with the historical aggregated transmission rate, where the historical aggregated transmission rate is the aggregated transmission rate within the previous time window. If the current aggregated transmission rate is greater than or equal to the historical aggregated transmission rate, the current parallelism is increased, a new target server is added from the set of available servers, and the sub-data blocks to be transmitted in the global task list are assigned to the newly added server. If the current aggregated transmission rate is less than the historical aggregated transmission rate, the current parallelism is reduced, and the server is released from the current target server. The sub-data blocks that were not successfully transmitted in the released server are returned to the global task list. If all sub-data blocks in the global task list are successfully transmitted, the received target sub-data blocks are retrieved from each target server, and each target sub-data block is spliced ​​and verified. If the verification is successful, the data transmission ends.

2. The method according to claim 1, characterized in that, The calculation of the aggregated transmission rate within the current time window specifically includes: Receive the sub-data block transmission completion confirmation information returned by the target server, obtain the unique identifier of the target sub-data block from the transmission completion confirmation information, and store the unique identifier of the target sub-data block in the data block identifier list corresponding to the current time window; When the current time window ends, the actual byte size of each identifier is retrieved from the list of data block identifiers, and the actual byte sizes corresponding to each identifier are accumulated to obtain the total amount of data successfully transmitted within the current time window. The current aggregated transmission rate is obtained by dividing the total data volume by the preset duration of the current time window.

3. The method according to claim 2, characterized in that, The step of increasing the current parallelism and adding a new target server from the available server set specifically includes: Divide the current aggregated transmission rate by the historical aggregated transmission rate to obtain the performance gain rate; The net gain rate is obtained by subtracting the performance gain rate from the base. When the net gain rate is greater than the preset growth trigger threshold, multiple historical aggregated transmission rates are combined to obtain a rate sequence. The performance fluctuation value is obtained by dividing the standard deviation of the rate sequence by the average value. The average packet loss rate and average latency jitter of each of the multiple target servers are obtained, and the average packet loss rate and average latency jitter are weighted and summed to obtain the comprehensive network loss value. The performance fluctuation value is calculated based on a preset first inverse proportional mapping function to obtain a performance stability index; the comprehensive network loss value is calculated based on a preset second inverse proportional mapping function to obtain a network health index. Multiply the base growth coefficient, the performance stability index, and the network health index to obtain the dynamic growth sensitivity coefficient for the current time window; multiply the net gain ratio, the current parallelism, and the dynamic growth sensitivity coefficient to obtain the theoretical increase. The larger value between the theoretical increase and the preset minimum increase step size is selected as the corrected increase, and the number of currently unoccupied available idle servers is counted from the available server set. The number of sub-data blocks that have not yet been allocated for transmission is counted from the global task list; The corrected increase amount, the number of available idle servers, and the number of sub-data blocks are compared, and the minimum value among the corrected increase amount, the number of available idle servers, and the number of sub-data blocks is selected as the final increase amount; Select the same number of servers as the final increase from the available server set as the new target servers, and add the final increase to the current parallelism to update it to the new current parallelism.

4. The method according to claim 3, characterized in that, The reduction of the current parallelism and the release of servers from the current target server specifically include: Divide the current aggregated transmission rate by the historical aggregated transmission rate to obtain the performance ratio; Subtracting the base from the performance ratio yields the performance degradation rate; calculating the performance fluctuation value based on a preset first proportional mapping function yields the system instability index; and calculating the comprehensive network loss value based on a preset second proportional mapping function yields the network congestion index. The dynamic reduction sensitivity coefficient of the current time window is obtained by weighted summing of the basic reduction coefficient, the system instability index, and the network congestion index. Multiply the performance degradation rate, the current parallelism, and the dynamic reduction sensitivity coefficient to obtain the theoretical reduction amount. Select the larger value between the theoretical reduction amount and the preset minimum reduction step size to obtain the corrected reduction amount. Multiply the current parallelism by the preset maximum reduction ratio to obtain the maximum allowable reduction amount; select the minimum value between the corrected reduction amount and the maximum allowable reduction amount to obtain the initial reduction amount; Subtract 1 from the current parallelism to obtain the maximum reduction quantity. Select the smaller value between the initial reduction quantity and the maximum reduction quantity as the final reduction quantity. Obtain the health score of each target server, and sort the target servers in ascending order based on the health scores; Select the same number of servers as the final reduction from the head of the sorting results as servers to be released, release the servers to be released, and return the untransmitted sub-data blocks undertaken by each of the servers to be released to the global task list; subtract the final reduction from the current parallelism to update the new current parallelism.

5. The method according to claim 1, characterized in that, The process of dividing the data to be transmitted into multiple sub-data blocks and then summarizing these sub-data blocks into a global task list specifically includes: Starting from the beginning position of the data to be transmitted, slide the window forward on the data to be transmitted in bytes; The rolling hash algorithm is used to calculate the rolling hash value of the data within the current sliding window; Determine whether the rolling hash value meets the splitting condition, or whether the number of bytes scanned in the current sliding window has reached the maximum block threshold; When the rolling hash value satisfies the splitting condition, or when the number of bytes scanned in the current sliding window reaches the maximum block threshold, the end position of the current sliding window is confirmed as the splitting point. The data to be transmitted is divided based on the marked split points to obtain multiple sub-data blocks. Each sub-data block is assigned a corresponding identifier and summarized into the global task list. The identifier includes a sequence identifier and a checksum of a single sub-data block.

6. The method according to claim 5, characterized in that, The process of concatenating and verifying each of the target sub-data blocks, and ending the data transmission after verification, specifically includes: Once all sub-data blocks in the global task list have been confirmed to have been successfully transmitted, a retrieval command is sent to all the target servers that participated in the transmission, and all target sub-data blocks and associated identifiers are obtained according to the retrieval command. Based on the identifier, the checksum of a pre-stored single sub-data block is queried from the global task list, and a hash calculation is performed on the received target sub-data block to obtain a real-time checksum; When the checksum of a single sub-data block is inconsistent with the real-time checksum, it is confirmed that the target sub-data block is corrupted. A retransmission request for the target sub-data block is triggered based on the identifier until the checksum passes. After all the retrieved target sub-data blocks have passed the block checksum, the target sub-data blocks are sorted in ascending order according to the sequence identifier. Based on the sorted order, the byte streams of each sub-data block are concatenated end to end to obtain the reconstructed data; The reconstructed data is hashed to obtain the reconstructed data checksum, and the data to be transmitted is hashed to obtain the overall data checksum. If the reconstructed data checksum is consistent with the overall data checksum, then the data transmission and splicing are confirmed to be error-free, and the current data transmission ends. If the reconstructed data checksum is inconsistent with the overall data checksum, then the data transmission or splicing is confirmed to have failed, and an anomaly warning is triggered based on the failure.

7. The method according to claim 1, characterized in that, The step of analyzing the real-time performance indicator information based on a preset health threshold, and selecting a set of available servers in an idle state from the server cluster based on the real-time performance indicator information, specifically includes: The load rate, memory usage, network egress bandwidth utilization, and network latency are obtained from the real-time performance metrics information. Obtain the overall data volume and data type of the data to be transmitted, and apply a set of weighting coefficients based on the overall data volume and data type; then assign the set of weighting coefficients to the load rate, the memory utilization rate, the network egress bandwidth utilization rate, and the network latency in sequence. For each server in the server cluster, the load rate, memory usage rate, network egress bandwidth utilization rate, and network latency are multiplied by their respective weight coefficients, and all products are summed to obtain the comprehensive health score of each server. When the overall health score is greater than or equal to the preset health threshold, the server is confirmed to be in the idle state. When the overall health score is less than the preset health threshold, the server is confirmed to be in a non-idle state. The servers in the server cluster that are in the idle state are aggregated to obtain the set of available servers.

8. An adaptive parallel data transmission system, characterized in that, The system includes a receiving unit, a processing unit, and a verification unit. The receiving unit receives the transmission request sent by the user and parses it to obtain the data to be transmitted. It divides the data to be transmitted into multiple sub-data blocks and summarizes the multiple sub-data blocks into a global task list. Based on broadcast probe requests, receive real-time performance metrics information returned by each server in the server cluster. The processing unit analyzes the real-time performance indicator information based on a preset health threshold and filters out the set of available servers in the idle state from the server cluster based on the real-time performance indicator information. An initial number of servers are selected from the available server set as target servers, and target sub-data blocks of the same number as the initial number are obtained from the global task list. The initial number is used as the current parallelism to establish a parallel transmission channel, and the target sub-data blocks are transmitted one-to-one to the target server. Calculate the aggregated transmission rate within the current time window, and compare the current aggregated transmission rate with the historical aggregated transmission rate, which is the aggregated transmission rate within the previous time window. If the current aggregated transmission rate is greater than or equal to the historical aggregated transmission rate, increase the current parallelism, add a new target server from the available server set, and allocate the sub-data blocks to be transmitted in the global task list to the new server. If the current aggregated transmission rate is less than the historical aggregated transmission rate, decrease the current parallelism, release the server from the current target server set, and return the sub-data blocks that were not successfully transmitted from the released server to the global task list. If all sub-data blocks in the global task list are successfully transmitted, the verification unit retrieves the received target sub-data blocks from each target server, splices and verifies each target sub-data block, and ends the data transmission after the verification is successful.

9. An electronic device, characterized in that, The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to cause the electronic device to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed, perform the method as described in any one of claims 1-7.

Images