File high-speed transmission system based on multilink aggregation

Abstract

The invention provides a file high-speed transmission method based on multi-link aggregation, and solves the problems of low transmission efficiency and insufficient stability of a traditional single link. The method comprises the following steps: detecting network link performance parameters, and screening available links; according to the number of links, the average bandwidth and # imgabs0 # limitation, dynamically segmenting a large file into adaptive data blocks, numbering the data blocks, and adding information such as a position to a head; an improved dynamic load balancing algorithm is adopted, link quality (including bandwidth utilization rate, delay, packet loss rate, stability and dynamic adjustment of weight along with a network environment) and data block priority (distinguished according to types and sizes and the weight is adjusted according to file characteristics) are synthesized, data blocks are distributed to different links for parallel transmission, and pre-distribution verification ensures link load safety; and recombining the data blocks by the receiving end according to the numbers to realize file recovery. The system significantly improves the transmission efficiency through multi-link aggregation and dynamic strategy adjustment.

Description

Technical Field

[0001] The present invention relates to the field of file transfer, and specifically to a high-speed file transfer system based on multi-link aggregation. Background Art

[0002] In the existing file transfer technology, taking FTP (File Transfer Protocol) as an example, when performing file transfer, it usually only uses a single network link for data transfer. When facing the scenario of large file transfer, this single-link transfer method has many problems. Due to the limited bandwidth of a single link, and there may be link congestion, instability, etc. in the network environment, resulting in slow file transfer speed and long transfer time, seriously affecting the user experience. For example, when transferring large-capacity files such as high-definition videos and large database files, using single-link FTP transfer may take several hours or even longer, and it is also prone to interruptions, data loss, etc. during the transfer process, greatly reducing the efficiency and reliability of file transfer. In addition, the single-link transfer method cannot make full use of multiple available link resources existing in the user's network environment, causing waste of network resources. Therefore, there is an urgent need for a method that can effectively solve the above problems and achieve high-speed and stable file transfer. Summary of the Invention

[0003] The purpose of the present invention is to provide a high-speed file transfer system based on multi-link aggregation to solve the problems raised in the above background art.

[0004] To achieve the above purpose, the present invention provides the following technical solution: A high-speed file transfer system based on multi-link aggregation, including the following steps: Step 1, Link Detection and Screening: Before the file transfer starts, the link detection module detects multiple network links existing in the user's network environment, and obtains the bandwidth, delay, packet loss rate, and network stability performance parameters of each link; according to the preset performance threshold, screen out the available links that meet the transfer requirements; Step 2, File Splitting: Split the large file to be transferred into several data blocks according to the preset rules; in order to ensure the integrity and transfer efficiency of the data blocks, split the file into data blocks with matching sizes according to the number and average bandwidth of the available links screened in Step 1; Step 3, Link Allocation and Transmission: Allocate each of the data blocks split in Step 2 to different available links for parallel transmission; specifically, use an improved dynamic load balancing algorithm for each of the split data blocks, and dynamically adjust the data block allocation strategy according to the real-time performance parameters of each link to obtain the final allocation plan, ensuring load balancing of each link and making full use of the transmission capabilities of each link; Step 4: Data Reception and Reassembly: At the receiving end, receive the data blocks transmitted through different links respectively, and reassemble them in the order of the original file according to the numbering information of the data blocks to restore the complete file. At the same time, check the received data blocks. If it is found that a data block is in error or missing, send a retransmission request to the sending end in a timely manner to ensure the accuracy and integrity of the file transmission.

[0005] Preferably, the specific implementation steps of the file separation are as follows: Calculate the initial data block size: After the file transmission task is started, first obtain the bandwidth information of all available links from Step 1 and calculate the average bandwidth , and the calculation formula is: , where represents the bandwidth of the th link, and is the total number of available links. At the same time, according to the protocol overhead and burst traffic factors in network transmission, set the empirical coefficient , where ranges from 0.7 to 0.9. Then, combine the total size of the file to be transmitted with the average bandwidth and the empirical coefficient to calculate the initial data block size ; Dynamic adjustment of data block size: After calculating the initial data block size, further analyze the latency and packet loss rate of each link. For links with high latency or large packet loss rate, reduce the data block size allocated to that link. The specific adjustment method is as follows: Let the latency of link be , and the packet loss rate be . Set the latency threshold and the packet loss rate threshold . If or , then adjust the data block size allocated to link to , where is the adjustment coefficient, and is determined according to the degree to which the latency and packet loss rate exceed the threshold; Data block segmentation and numbering: Divide the file according to the determined data block size. Starting from the beginning position of the file, divide it into multiple data blocks in sequence, and record the size of each data block as ; Assign a unique number to each data block, and the numbers increase sequentially from 1 according to the segmentation order. At the same time, add header information to each data block. The header information includes key information such as the data block number, data block size, and starting position in the original file, which facilitates data reassembly at the receiving end; Data block check information generation: To ensure the integrity of data blocks during transmission, a hash algorithm is used to calculate each data block, generating the corresponding hash value, and adding the hash value to the header information of the data block; after receiving the data block at the receiving end, the integrity of the data block is judged by recalculating the hash value and comparing it with the received hash value.

[0006] Preferably, based on an improved dynamic load balancing algorithm, according to the real-time performance parameters of each link, the data block allocation strategy is dynamically adjusted, and its specific implementation logic is as follows: First, construct a comprehensive link quality evaluation model , where is composed of bandwidth utilization , real-time delay , packet loss rate and link stability indicators, and the specific calculation formula is as follows: , where represents the quality score of the th link, and the higher the score, the better the link quality; is the weight coefficient, and the specific value of the weight coefficient is dynamically adjusted according to the network environment, and , is the minimum delay value among all available links, used to normalize the delay index; is the link stability index, obtained by calculating the variance of the fluctuation of the link performance parameters, and the smaller the variance, the higher the stability; Determination of data block transmission priority: Different transmission priorities are assigned to each data block after segmentation in step 2 according to the type and importance of the data block. High priority is assigned to critical data blocks, and low priority is assigned to ordinary data blocks. The priority calculation formula is as follows: , where represents the transmission priority of the th data block, represents the data block type factor, with a value of 1 for critical data blocks and 0.5 for ordinary data blocks; represents the data block size factor, obtained by calculation; is the weight coefficient, where , and the specific value is dynamically adjusted according to the file characteristics, and the weight coefficient corresponding to each file characteristic is set in advance; Dynamic weight adjustment algorithm: Based on the link quality score and data block priority, a dynamic weight adjustment algorithm is proposed to determine the number of data blocks allocated to each link, that is, the allocation scheme is obtained; the specific calculation formula of the dynamic weight adjustment algorithm is as follows: , where Indicates the number of data blocks allocated to the th link; Indicates the total number of data blocks to be allocated; Indicates the total number of available links; Is the link-data block adaptation factor. When the link 's (maximum transmission unit) fully accommodates the data block When , otherwise ; Adjust the allocation scheme: Verify the pre-allocation of data blocks in the obtained allocation scheme, and check whether the data block allocation of each link in the initial allocation scheme meets its bandwidth and Limit; if there is a link exceeding its carrying capacity, fine-tune the allocation scheme to obtain the final allocation scheme to ensure that all links can receive and transmit data blocks within the safe load range; Data block scheduling and transmission: The data transmission module sends the data blocks to the corresponding links according to the final allocation scheme and monitors the link status in real time during the transmission process.

[0007] Preferably, for The specific steps for dynamically adjusting the weight coefficient are as follows: Analysis of network environment characteristics: The link detection module continuously monitors the network environment and collects network status data every 1 minute, including network type (such as enterprise private network, home broadband, mobile cellular network), current network load (evaluated by monitoring the total bandwidth utilization rate of all available links), network stability (statistics of indicators such as the number of link interruptions and delay fluctuation amplitude per unit time); Initial setting of weight coefficient: According to the monitored network environment characteristics, perform Initial setting of the weight coefficient, where different Weight coefficients are set with different setting values under different network environment characteristics; Judgment of dynamic adjustment trigger conditions: Re-evaluate the network environment characteristics every 5 minutes and calculate the difference value between the current network state and the previous evaluation; When one of the following situations occurs, trigger the dynamic adjustment of the weight coefficient: The network type changes (such as switching from home broadband to mobile cellular network); The network load changes by more than 20% (for example, the total bandwidth utilization rate rises from 40% to 60%); The network stability indicators (such as the number of link interruptions and delay fluctuation amplitude) change by more than the preset threshold (such as the number of link interruptions increases from 1 time / hour to 3 times / hour); Weight coefficient adjustment strategy: When the adjustment condition is triggered, according to the new network environment characteristics, set the corresponding Weight coefficient to the setting value under this network environment characteristic for weight coefficient adjustment; Post - adjustment verification and optimization: After completing the adjustment of the weight coefficients, a 10 - minute verification period is carried out; during the verification period, compare the link quality scores before and after the adjustment of the weight coefficients for changes; if it is found that the link quality score shows abnormal fluctuations after adjustment, the weight coefficients are adjusted according to the actual situation until the optimal transmission effect is achieved.

[0008] Preferably, the specific implementation logic for verifying the pre - allocation of data blocks in the obtained allocation scheme is as follows: Obtain link parameters: After the link allocation module calculates the initial allocation scheme using the dynamic weight adjustment algorithm, immediately obtain all available link parameters from the link detection module; the performance parameters include the real - time bandwidth of each link and the maximum transmission unit , the currently used bandwidth and the link quality score ; Calculate the theoretical carrying capacity of each link: According to the obtained link parameters, calculate the number of data blocks and the data volume that each link can carry in the current state respectively. For the calculation of the number of data blocks, first calculate the data volume that the link can transmit within a unit time , where t is the unit time, and then combine with the average size of the data blocks in step to obtain the number of data blocks that can be carried ; at the same time, according to the maximum transmission unit limit, calculate the maximum number of data blocks that the link can carry , and then take the smaller value between the number of data blocks that can be carried and the maximum number of data blocks that can be carried as the theoretical carrying data block number of the link; Evaluate the initial allocation scheme: Compare the number of data blocks allocated to each link in the initial allocation scheme with the calculated theoretical carrying data block number; specifically, and , then it is considered that the allocation of this link meets the bandwidth and limitations; if there is or situation, then it is determined that this link exceeds the carrying capacity, and the allocation scheme is fine - tuned; Allocation scheme fine - tuning strategy: When it is found that there is a link exceeding the carrying capacity, adjust according to the link quality score and the data block priority . Specifically, in the order from low to high of the data block priority, remove the data blocks from the overloaded link one by one, and recalculate the carrying situation of the target link to ensure that the target link is still within the safe load range after receiving the transferred data blocks. At the same time, during the process of transferring data blocks, preferentially select the adaptation factor Use the link with an ID of 1 as the transfer target to reduce data fragmentation and recombination overhead; Determine the final allocation plan: After fine-tuning, check the data block allocation of all links again to ensure that the data block allocation of each link meets the bandwidth and limitations; if all links pass the verification, determine the adjusted allocation plan as the final allocation plan, and the data transfer module schedules the data blocks to the corresponding links for transmission according to the final allocation plan; if there are still links that do not meet the conditions, repeat the fine-tuning operation of the allocation plan fine-tuning strategy until all links meet the requirements.

[0009] Preferably, a file high-speed transmission system based on multi-link aggregation, characterized in that it includes: a link detection module for detecting multiple links in the network environment, obtaining the performance parameters of each link, and performing link screening; a file segmentation module for segmenting a large file to be transmitted to generate multiple data blocks: a link allocation module for allocating each segmented data block to different available links and dynamically adjusting the allocation strategy according to the link performance; a data transfer module for parallel transmission of data blocks through the allocated links: a data reception and recombination module for receiving data blocks, performing recombination and verification to ensure the integrity and accuracy of file transmission.

[0010] Compared with the prior art, the beneficial effects of the present invention are: significantly improving the file transmission efficiency, greatly shortening the transmission time, breaking through the single-link bandwidth limitation by segmenting a large file into multiple data blocks and allocating them to multiple available links for parallel transmission. For example, when transmitting a 300MB file on 3 links with an average bandwidth of 21Mbps, the transmission time can be shortened to about 1 / 3 of the single-link transmission. Compared with the traditional single-link FTP transmission scenario that takes several hours, the present invention can compress the transmission time to the minute level or even shorter. The improved dynamic load balancing algorithm realizes the precise matching of data blocks and links through a link quality comprehensive evaluation model (integrating bandwidth utilization rate, delay, packet loss rate, stability) and a data block priority strategy. For example, key data blocks (such as file metadata) are preferentially allocated to high-quality links, and ordinary data blocks are dynamically adjusted according to the real-time load of the links to avoid link congestion and further improve the overall transmission efficiency. The data block size is dynamically adjusted according to the link to avoid data fragmentation overhead; Dynamic response to link stability: For links with high latency or large packet loss rate, automatically reduce the size of the allocated data block (such as the original data block size × (1 - ), = 0.1 - 0.3), reducing the probability of single-block data transmission failure. For example, on a link with an excessive packet loss rate, adjust the data block size from 80MB to 56MB ( ), which can reduce the link transmission error rate by more than 40%. Link quality assessment weight ( ): Dynamically adjust the weights of each index according to the network environment (such as enterprise private network, home broadband); Multi-link aggregation and load balancing: Make full use of multiple available links (such as broadband + mobile network) in the user's network to avoid single-link idleness. Through the dynamic weight adjustment algorithm, monitor the link load in real time (such as triggering adjustment when the bandwidth utilization rate exceeds 80%), and transfer the data blocks of the overloaded link to the low-load link. Brief Description of the Drawings

[0011] Figure 1 is a schematic flow chart of the method of the present invention; Figure 2 is a schematic flow chart of the file separation implementation steps of the present invention; Figure 3 is a schematic flow chart of the improved dynamic load balancing algorithm implementation of the present invention; Figure 4 is a schematic diagram of the system structure of the present invention. Detailed Embodiment

[0012] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0013] Embodiment 1 Please refer to Figures 1-3 , the present invention provides a technical solution: A file high-speed transmission system based on multi-link aggregation, including the following steps: Step 1, Link detection and screening: Before the file transmission starts, the link detection module detects multiple network links existing in the user's network environment, and obtains the bandwidth, delay, packet loss rate and network stability performance parameters of each link; According to the preset performance threshold, screen out the available links that meet the transmission requirements; Step 2, File segmentation: Divide the large file to be transmitted into several data blocks according to the preset rules; In order to ensure the integrity and transmission efficiency of the data blocks, divide the file into data blocks with matching sizes according to the number and average bandwidth of the available links screened in Step 1; The specific implementation steps are as follows: Calculate the initial data block size: After the file transmission task starts, first obtain the bandwidth information of all available links from Step 1, and calculate the average bandwidth , and the calculation formula is: , where represents the The bandwidth of one link is the total number of available links; at the same time, according to the protocol overhead and burst traffic factors in network transmission, an empirical coefficient is set, where ranges from 0.7 to 0.9; then, the total size of the file to be transmitted is combined with the average bandwidth and the empirical coefficient to calculate the initial data block size ; for example, if there are 3 available links with bandwidths of 20Mbps, 25Mbps, and 18Mbps respectively, then the average bandwidth is set, and the empirical coefficient is set. If the total file size is 300MB, after conversion it is , then the initial data block size , that is, approximately 80M.

[0014] Dynamic adjustment of data block size: After calculating the initial data block size, further analyze the delay and packet loss rate of each link. For links with high delay or large packet loss rate, reduce the data block size allocated to that link; the specific adjustment method is as follows: Let the delay of link be , and the packet loss rate be . Set the delay threshold and the packet loss rate threshold . If or , then adjust the data block size allocated to link to , where is the adjustment coefficient, which is determined according to the degree to which the delay and packet loss rate exceed the threshold; through this dynamic adjustment, avoid large - scale data transmission failure caused by unstable links and improve the reliability of transmission.

[0015] Data block splitting and numbering: According to the determined data block size, perform a splitting operation on the file. Starting from the starting position of the file, divide it into multiple data blocks in sequence, then the size of each data block is recorded as ; assign a unique number to each data block, and the number increases sequentially from 1 according to the splitting order; at the same time, add header information to each data block, and the header information includes key information such as the data block number, data block size, and starting position in the original file, which is convenient for the receiving end to perform data recombination; Data block check information generation: To ensure the integrity of data blocks during transmission, a hash algorithm is used to calculate each data block to generate a corresponding hash value, and the hash value is added to the header information of the data block. After the receiving end receives the data block, it determines whether the data block is complete by recalculating the hash value and comparing it with the received hash value.

[0016] It should be noted that after splitting the large file into multiple data blocks through the file splitting in step 2, the advantages of multi-link aggregation transmission can be fully utilized to achieve parallel transmission of data blocks on different links. Compared with transmitting a large file on a single link, simultaneously transmitting data blocks on multiple links can significantly shorten the transmission time. By calculating and determining an appropriate data block size based on the number of available links and the average bandwidth, it can ensure that each data block can be efficiently transmitted on the link, avoiding the situation of long-term occupation of the link due to overly large data blocks or excessive transmission overhead due to overly small data blocks, making more full and reasonable use of network resources and further improving the overall transmission efficiency. At the same time, file splitting enhances the reliability of file transmission: The data blocks are relatively small. During transmission, if a temporary failure, congestion, or packet loss occurs on a certain link, only the corresponding data block transmission will be affected, and the entire large file transmission will not fail. The receiving end can initiate a retransmission request for the lost or incorrect data blocks based on the data block number and check information. Compared with retransmitting the entire file when a problem occurs during large file transmission, this method can significantly reduce the risk of data loss and improve the integrity and accuracy of file transmission. For links with relatively high latency or large packet loss rate, dynamically adjust the data block size allocated to this link. When smaller data blocks are transmitted on unstable links, they are less affected by network fluctuations, reducing the probability of data transmission errors. For example, on a link with a high packet loss rate, appropriately reducing the data block size can reduce the possibility of retransmission of a single data block due to packet loss and ensure the stability of file transmission. It can also improve transmission flexibility and adaptability: That is, the characteristics such as bandwidth, latency, (maximum transmission unit) of different network links vary. The file splitting step can make the data blocks better adapt to the characteristics of each link by dynamically adjusting the data block size. For example, for smaller links, allocate data blocks of appropriate size to avoid unnecessary fragmentation and recombination of data during transmission, reduce transmission overhead, and improve the transmission efficiency and stability of the link. Whether it is different types of large files such as high-definition videos and large database files, or file transmission scenarios in different network environments, the file splitting step can reasonably split and process data blocks according to the actual situation. By flexibly adjusting the data block size, allocation strategy, etc., it can meet diverse file transmission requirements and make the file transmission method of the present invention have a wider applicability.

[0017] Step 3, Link Allocation and Transmission: Allocate each of the data blocks segmented in Step 2 to different available links for parallel transmission; specifically, adopt an improved dynamic load balancing algorithm for each of the segmented data blocks, and dynamically adjust the allocation strategy of the data blocks according to the real-time performance parameters of each link to obtain the final allocation plan, ensuring load balancing for each link and making full use of the transmission capabilities of each link; Among them, based on the improved dynamic load balancing algorithm, according to the real-time performance parameters of each link, the allocation strategy of the data blocks is dynamically adjusted, and its specific implementation logic is as follows: First, construct a comprehensive evaluation model for link quality , where is composed of bandwidth utilization , real-time delay , packet loss rate and link stability indicators. The specific calculation formula is as follows: , where represents the quality score of the rd link. The higher the score, the better the link quality; is the weight coefficient, and the specific value of the weight coefficient is dynamically adjusted according to the network environment, and , is the minimum delay value among all available links, which is used to normalize the delay index; is the link stability index, which is obtained by calculating the variance of the fluctuations of the link performance parameters. The smaller the variance, the higher the stability; Determination of Data Block Transmission Priority: Assign different transmission priorities to each of the data blocks segmented in Step 2 according to their types and importance. High priority is assigned to critical data blocks, and low priority is assigned to ordinary data blocks. The priority calculation formula is as follows: , where represents the transmission priority of the th data block, represents the data block type factor, with a value of 1 for critical data blocks and 0.5 for ordinary data blocks; represents the data block size factor, which is calculated by ; is the weight coefficient, where , and the specific value is dynamically adjusted according to the file characteristics, and the weight coefficient corresponding to each file characteristic is set in advance; Dynamic Weight Adjustment Algorithm: Based on the link quality score and data block priority, a dynamic weight adjustment algorithm is proposed to determine the number of data blocks allocated to each link, that is, to obtain the allocation plan; the specific calculation formula of the dynamic weight adjustment algorithm is as follows: , where represents the number of data blocks allocated to the th link; represents the total number of data blocks to be allocated; represents the total number of available links; is the link-data block adaptation factor. When the link 's (maximum transmission unit) fully accommodates the data block when , otherwise ; Adjust the allocation scheme: Verify the pre-allocation of data blocks in the obtained allocation scheme, and check whether the data block allocation of each link in the initial allocation scheme meets its bandwidth and limits; If there is a link exceeding its carrying capacity, fine-tune the allocation scheme to obtain the final allocation scheme to ensure that all links can receive and transmit data blocks within the safe load range; Data block scheduling and transmission: The data transmission module sends the data blocks to the corresponding links according to the final allocation scheme and monitors the link status in real time during the transmission process.

[0018] It should be noted that the improved dynamic load balancing algorithm more accurately reflects the actual transmission capacity of the link by comprehensively considering four dimensions: bandwidth utilization, latency, packet loss rate, and stability; It can also assign different priorities according to the importance and type of data blocks to ensure the priority transmission of critical data and improve the overall transmission reliability; And a fast response mechanism is established, which can adjust the allocation strategy in a timely manner when the link performance changes to adapt to the dynamic changes of the network environment; At the same time, by introducing the link-data block adaptation factor , avoid the overhead of data fragmentation and recombination caused by mismatch and improve the transmission efficiency.

[0019] It should be noted that the specific steps for dynamically adjusting the weight coefficient are as follows: Network environment feature analysis: The link detection module continuously monitors the network environment and collects network status data every 1 minute, including network type (such as enterprise private network, home broadband, mobile cellular network), current network load (evaluated by monitoring the total bandwidth utilization of all available links), network stability (statistics of indicators such as the number of link interruptions and the amplitude of latency fluctuations per unit time); For example, in an enterprise private network environment, the network stability is usually high, but there may be fixed bandwidth limitations; While the home broadband network may have large bandwidth fluctuations and unstable latency during peak hours; Initial setting of weight coefficient: According to the monitored network environment features, perform the initial setting of the weight coefficient, where under different network environment features ​The weight coefficients are correspondingly set with different setting values; if it is detected that the network environment is an enterprise private network, and the network load is low and the stability is high, then the initial setting (bandwidth utilization weight), (delay weight), (packet loss rate weight), (link stability weight) are emphasized to highlight the influence of bandwidth utilization and stability on link quality evaluation; if it is the peak period of a home broadband network, and and and can be set to highlight the importance of bandwidth utilization and delay.

[0020] Dynamic adjustment trigger condition judgment: Re-evaluate the network environment characteristics every 5 minutes, and calculate the difference value between the current network state and the previous evaluation; when one of the following situations occurs, trigger the dynamic adjustment of the weight coefficients: The network type changes (such as switching from a home broadband network to a mobile cellular network); the network load changes by more than 20% (for example, the total bandwidth utilization rate increases from 40% to 60%); the network stability indicators (such as the number of link interruptions, the amplitude of delay fluctuations) change by more than a preset threshold (such as the number of link interruptions increases from 1 time per hour to 3 times per hour); Weight coefficient adjustment strategy: After the adjustment condition is triggered, according to the new network environment characteristics, set the corresponding weight coefficient to the setting value under this network environment characteristic for weight coefficient adjustment; for example, if the network switches from an enterprise private network to a mobile cellular network, considering that the mobile network bandwidth is relatively unstable and the delay is high, adjust the weights to and and and to increase the proportion of the delay index in the link quality evaluation; if the increase in network load leads to intensified bandwidth competition, then appropriately increase the value and reduce the value to pay more attention to the influence of bandwidth utilization on link quality.

[0021] Verification and optimization after adjustment: After completing the weight coefficient adjustment, conduct a 10-minute verification period; during the verification period, compare the change in the link quality score before and after the weight coefficient adjustment ; if it is found that the link quality score shows abnormal fluctuations after the adjustment, then adjust the weight coefficients according to the actual situation until the optimal transmission effect is achieved.

[0022] For the weight coefficient calculation of data block priority the dynamic adjustment steps are briefly described as follows: File characteristic analysis: When the file splitting module obtains the information of the file to be transmitted, the file characteristics are analyzed simultaneously. The analysis content includes file type (such as video file, database file, text file), file structure (whether it contains important metadata, index information, etc.), and file usage scenario (such as working files for emergency transmission, daily backup files). For example, for video files, the continuity and integrity of data blocks are more important; while for database files, the transmission priority of metadata and index information is higher; Initial setting of weight coefficients: The initial setting of weight coefficients is carried out according to file characteristics. If the file is a database file containing key business data, the initial setting (weight of data block type factor), (weight of data block size factor) are used to highlight the impact of data block type on priority and ensure the priority transmission of metadata and index information; if it is an ordinary video file, then set 、 to balance the impact of data block type and size on priority.

[0023] During the file transmission process, the file transmission status and changes in user requirements are monitored in real time. When one of the following situations occurs, the dynamic adjustment of weight coefficients is triggered: A priority adjustment instruction from the user is received during the file transmission process (such as the user manually marks some data blocks as urgent transmission); The file transmission progress reaches a specific node (such as when 20% or 50% of the transmission is completed, it is judged whether adjustment is needed according to the feedback of the transmitted data blocks); Special requirements related to the file type appear (such as real-time preview is required during the transmission of a video file, and at this time, the priority of key frame data blocks needs to be increased); Weight coefficient adjustment strategy: After the adjustment condition is triggered, the weight coefficients are adjusted according to the specific situation. If the user marks some data blocks as urgent transmission, will be increased to 0.8 or even higher to ensure the priority transmission of these data blocks; if it is found that some data blocks are too large and cause low transmission efficiency when the file is transmitted to 50%, then value is appropriately reduced, and value is increased to make the system pay more attention to the impact of data block type on priority and give priority to transmitting important type data blocks.

[0024] The specific implementation logic for the pre-allocation verification of data blocks in the obtained allocation scheme is as follows: Obtain link parameters: After the link allocation module calculates the initial allocation scheme using the dynamic weight adjustment algorithm, immediately obtain all available link parameters from the link detection module; The performance parameters include the real-time bandwidth of each link 、maximum transmission unit , current used bandwidth and link quality scores ; Calculate the theoretical carrying capacity of each link: According to the obtained link parameters, calculate the number of data blocks and the amount of data that each link can carry in the current state. For the calculation of the number of data blocks, first calculate the amount of data that the link can transmit per unit time , where t is the unit time, and then combine it with the average size of the data block in step to obtain the number of data blocks that can be carried ; At the same time, according to the maximum transmission unit limit, calculate the maximum number of data blocks that the link can carry , and then take the smaller value of the number of data blocks that can be carried and the maximum number of data blocks that can be carried as the theoretical carrying data block number of the link; Evaluate the initial allocation plan: Compare the number of data blocks allocated to each link in the initial allocation plan with the calculated theoretical carrying data block number; specifically and , it is considered that the allocation of this link meets the bandwidth and limitations; if there is or situation, it is determined that this link exceeds the carrying capacity, and the allocation plan is fine-tuned; Allocation plan fine-tuning strategy: When it is found that there is a link that exceeds the carrying capacity, adjust according to the link quality score and data block priority . Specifically, in the order from low to high data block priority, remove the data blocks from the overloaded link one by one, and recalculate the carrying situation of the target link to ensure that the target link is still within the safe load range after receiving the transferred data blocks. At the same time, during the process of transferring data blocks, give priority to selecting the link with the adaptation factor of 1 as the transfer target to reduce data fragmentation and recombination overhead; Determine the final allocation plan: After fine-tuning, check the data block allocation of all links again to ensure that the data block allocation of each link meets the bandwidth and limitations; if all links pass the verification, determine the adjusted allocation plan as the final allocation plan, and the data transmission module schedules the data blocks to the corresponding links for transmission according to the final allocation plan; if there are still links that do not meet the conditions, repeat the fine-tuning operation of the allocation plan fine-tuning strategy until all links meet the requirements.

[0025] Step 4: Data Reception and Reassembly: At the receiving end, the data blocks transmitted through different links are received respectively, and according to the numbering information of the data blocks, they are reassembled in the order of the original file to restore the complete file. At the same time, the received data blocks are verified. If it is found that a data block has an error or is lost, a retransmission request is sent to the sending end in a timely manner to ensure the accuracy and integrity of the file transmission.

[0026] Embodiment 2 Please refer to Figure 4 , a high-speed file transmission system based on multi-link aggregation, includes: a link detection module, which is used to detect multiple links in the network environment, obtain the performance parameters of each link, and perform link screening; a file segmentation module, which is used to segment the large file to be transmitted to generate multiple data blocks; a link allocation module, which is used to allocate each segmented data block to different available links and dynamically adjust the allocation strategy according to the link performance; a data transmission module, which is used to perform parallel transmission of data blocks through the allocated links; a data reception and reassembly module, which is used to receive data blocks, perform reassembly and verification to ensure the integrity and accuracy of file transmission.

[0027] The present invention discloses a high-speed file transmission method and system based on multi-link aggregation, aiming to solve the problems of low efficiency, poor reliability and insufficient utilization of network resources in traditional single-link file transmission. The method first obtains performance parameters such as bandwidth, delay, packet loss rate and stability of multiple links in the user's network environment through the link detection module, and screens out the available links that meet the preset performance thresholds; then dynamically segments the large file to be transmitted into adapted data blocks according to factors such as the number of available links, average bandwidth and limitations, generates numbers, location information and hash check values for each data block to ensure integrity; then uses an improved dynamic load balancing algorithm to comprehensively consider the link quality score (integrating bandwidth utilization rate, normalized delay, packet loss rate, stability, and the weight coefficient is dynamically adjusted according to the network environment) and the data block priority (distinguishing critical data from ordinary data, and the weight coefficient is dynamically adjusted according to the file characteristics), and allocates the data blocks to different links for parallel transmission. Before allocation, pre-allocation verification is carried out to ensure the safe load under the link bandwidth and limitations; finally, at the receiving end, the data blocks are reassembled according to the numbers, and accurate retransmission of error data blocks is realized through hash verification. The system significantly improves the speed, stability and flexibility of large file transmission through multi-link aggregation, dynamic load balancing, data block-level reliability design and network environment adaptive mechanism.

[0028] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirits of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A file high-speed transmission method based on multi-link aggregation, characterized in that, It includes the following steps: Step 1, Link Detection and Screening: Before the file transfer starts, the link detection module detects multiple network links existing in the user's network environment, obtains the bandwidth, latency, packet loss rate, and network stability performance parameters of each link; according to the preset performance thresholds, filters out the available links that meet the transmission requirements; Step 2, File Splitting: Split the large file to be transmitted into several data blocks according to the preset rules; in order to ensure the integrity and transmission efficiency of the data blocks, split the file into data blocks with matching sizes according to the number and average bandwidth of the available links screened in Step 1; Step 3, Link Allocation and Transmission: Allocate the data blocks split in Step 2 to different available links for parallel transmission; specifically, adopt an improved dynamic load balancing algorithm for the split data blocks, dynamically adjust the data block allocation strategy according to the real-time performance parameters of each link, and obtain the final allocation plan to ensure the load balance of each link and make full use of the transmission capabilities of each link; Step 4, Data Reception and Reassembly: At the receiving end, receive the data blocks transmitted through different links respectively, and reassemble them in the order of the original file according to the number information of the data blocks to restore the complete file; at the same time, check the received data blocks, if it is found that the data blocks are in error or lost, send a retransmission request to the sending end in time to ensure the accuracy and integrity of the file transmission.

2. The method for high-speed file transmission based on multi-link aggregation according to claim 1, wherein: The specific implementation steps of the file separation are as follows: Calculate the initial data block size: After the file transfer task is started, first obtain the bandwidth information of all available links from Step 1 and calculate the average bandwidth. , and the calculation formula is: , where represents the bandwidth of the th link, is the total number of available links; at the same time, according to the protocol overhead and burst traffic factors in network transmission, set the empirical coefficient , where ranges from 0.7 to 0.9; then combine the total size of the file to be transmitted with the average bandwidth and the empirical coefficient, and calculate the initial data block size . Dynamic adjustment of data block size: After calculating the initial data block size, further analyze the latency and packet loss rate of each link. For links with high latency or large packet loss rate, reduce the data block size allocated to that link. The specific adjustment method is as follows: Let the latency of link be , the packet loss rate be , set the latency threshold and the packet loss rate threshold . If or , then adjust the data block size allocated to link to , where is the adjustment coefficient, determined according to the degree to which the latency and packet loss rate exceed the threshold. Data block splitting and numbering: According to the determined data block size, the file is split. Starting from the beginning position of the file, multiple data blocks are sequentially divided. Then, the size of each data block is recorded as ; A unique number is assigned to each data block , and the numbers increase sequentially from 1 according to the splitting order. At the same time, header information is added to each data block. The header information includes key information such as the data block number, data block size, and starting position in the original file, which facilitates data recombination at the receiving end; Generation of Data Block Check Information: To ensure the integrity of the data blocks during transmission, use the hash algorithm to calculate each data block to generate the corresponding hash value, and add the hash value to the header information of the data block; After the receiving end receives the data block, judge whether the data block is complete by recalculating the hash value and comparing it with the received hash value.

3. A file high-speed transmission method based on multi-link aggregation according to claim 1, characterized in that: Based on the improved dynamic load balancing algorithm, dynamically adjust the data block allocation strategy according to the real-time performance parameters of each link, and its specific implementation logic is as follows: First, construct a comprehensive evaluation model for link quality , where is composed of bandwidth utilization , real-time delay , packet loss rate and link stability indicators. The specific calculation formula is as follows: , where represents the quality score of the th link. The higher the score, the better the link quality; is the weight coefficient, and the specific value of the weight coefficient is dynamically adjusted according to the network environment, and ; is the minimum delay value among all available links, which is used to normalize the delay indicator; is the link stability indicator, which is obtained by calculating the variance of the fluctuations of the link performance parameters. The smaller the variance, the higher the stability; Determination of Data Block Transmission Priority: Assign different transmission priorities to the data blocks split in Step 2 according to their types and importance, assign high priority to critical data blocks, and assign low priority to ordinary data blocks; The priority calculation formula is as follows: , where represents the transmission priority of the th data block, represents the data block type factor, with a value of 1 for critical data blocks and 0.5 for ordinary data blocks; represents the data block size factor, which is calculated through ; is the weight coefficient, where , and the specific value is dynamically adjusted according to the file characteristics, and the weight coefficient corresponding to each file characteristic is set in advance; Dynamic Weight Adjustment Algorithm: Based on link quality scoring and data block priority, a dynamic weight adjustment algorithm is proposed to determine the number of data blocks allocated to each link, that is, to obtain the allocation scheme. The specific calculation formula of the dynamic weight adjustment algorithm is as follows: , where represents the number of data blocks allocated to the th link; represents the total number of data blocks to be allocated; represents the total number of available links; is the link-data block adaptation factor. When the link fully accommodates the data block , otherwise at this time , ; Adjust the allocation plan: Verify the pre-allocation of data blocks in the obtained allocation plan, and check whether the data block allocation of each link in the initial allocation plan meets its bandwidth and limit; if there is a link exceeding its carrying capacity, fine-tune the allocation plan to obtain the final allocation plan, ensuring that all links can receive and transmit data blocks within the safe load range; Scheduled Transmission of Data Blocks: The data transmission module sends the data blocks to the corresponding links according to the final allocation plan and monitors the link status in real time during the transmission process.

4. A file high-speed transmission method based on multi-link aggregation according to claim 3, characterized in that: For The specific steps for dynamically adjusting the weight coefficient are as follows: Analysis of Network Environment Characteristics: The link detection module continuously monitors the network environment and collects network status data every 1 minute, including network type, current network load, and network stability; Initial setting of weight coefficient: According to the network environment characteristics obtained by monitoring, perform the initial setting of the weight coefficient, where different weight coefficients are set with different values under different network environment characteristics; Judgment of Dynamic Adjustment Trigger Conditions: Re-evaluate the network environment characteristics every 5 minutes, calculate the difference value between the current network state and the last evaluation; when one of the following situations occurs, trigger the dynamic adjustment of the weight coefficient: the network type changes; the network load changes by more than 20%; the network stability index changes by more than the preset threshold; Weight coefficient adjustment strategy: After the adjustment condition is triggered, according to the new network environment characteristics, the corresponding weight coefficient is set to the set value under the network environment characteristics for weight coefficient adjustment; Post-adjustment verification and optimization: After the weight coefficient adjustment is completed, a 10-minute verification period is carried out; during the verification period, compare the link quality scores before and after the weight coefficient adjustment for changes; if abnormal fluctuations are found in the link quality score after adjustment, the weight coefficient is adjusted according to the actual situation until the optimal transmission effect is achieved.

5. The method for high-speed file transmission based on multi-link aggregation according to claim 3, wherein: The specific implementation logic for the pre-allocation verification of the data blocks in the obtained allocation plan is as follows: Obtain link parameters: After the link allocation module calculates the initial allocation scheme using the dynamic weight adjustment algorithm, immediately obtain all available link parameters from the link detection module; the performance parameters include the real-time bandwidth of each link , the maximum transmission unit , the currently used bandwidth and the link quality score ; Calculate the theoretical carrying capacity of each link: According to the obtained link parameters, calculate the number of data blocks and the data volume that each link can carry in the current state respectively. For the calculation of the number of data blocks, first calculate the data volume that the link can transmit within a unit time , where t is the unit time, and then combine it with the average size of the data blocks in step to obtain the number of data blocks that can be carried ; At the same time, according to the maximum transmission unit limit, calculate the maximum number of data blocks that the link can carry , and then take the smaller value of the number of data blocks that can be carried and the maximum number of data blocks that can be carried as the theoretical carrying data block number of the link; Evaluate the initial allocation plan: the number of data blocks allocated to each link in the initial allocation plan is compared with the theoretically calculated number of data blocks that can be carried; specifically, and , it is considered that the allocation of this link meets the bandwidth and restrictions; if there is or cases, it is determined that the link exceeds its carrying capacity, and the allocation plan is fine-tuned; Allocation Scheme Fine - Tuning Strategy: When it is found that there is a link exceeding its carrying capacity, adjust according to the link quality score and the data block priority Specifically, in the order from low to high data block priority, remove data blocks from the overloaded link one by one, and recalculate the carrying situation of the target link to ensure that the target link is still within the safe load range after receiving the transferred data blocks. At the same time, during the process of transferring data blocks, preferentially select the link with an adaptation factor of 1 as the transfer target to reduce the data fragmentation and recombination overhead; Determine the final allocation plan: After fine-tuning, check the data block allocation of all links again to ensure that the data block allocation of each link meets the bandwidth and restrictions; if all links pass the verification, determine the adjusted allocation plan as the final allocation plan, and the data transmission module schedules the data blocks to the corresponding links for transmission according to the final allocation plan; if there are still links that do not meet the conditions, repeat the fine-tuning operation of the allocation plan fine-tuning strategy until all links meet the requirements.

6. A file high-speed transmission system based on multi-link aggregation according to any one of claims 1-5, characterized in that, It includes: A link detection module, which is used to detect multiple links in the network environment, obtain the performance parameters of each link, and perform link screening; A file splitting module for splitting large files to be transmitted into multiple data blocks: A link allocation module for allocating each split data block to different available links and dynamically adjusting the allocation strategy according to link performance; A data transmission module for parallel transmission of data blocks through the allocated links: A data receiving and recombination module for receiving data blocks, performing recombination and verification to ensure the integrity and accuracy of file transmission.

Images