A method for optimizing transmission of super-large text based on dynamic block compression and routing

Abstract

This invention relates to an optimization method for ultra-large text transmission based on dynamic block compression and routing, belonging to the field of ultra-large text transmission technology. The method includes: constructing a semantic relationship graph from the ultra-large text through semantic analysis; dividing the ultra-large text into logical segments based on semantic relevance; dynamically adjusting the size of the block data based on the access frequency of the logical segments; matching the optimal compression algorithm to the block data according to the data characteristics of each block data; and dynamically predicting the transmission network congestion status based on real-time monitoring of the current network status. If there is no congestion, the compressed data blocks are transmitted to the target node one by one according to the original path; if there is congestion, the optimal path is replanned, the SDN route is updated, and the compressed data blocks are transmitted to the target node according to the updated path. This invention improves the efficiency of ultra-large text transmission, optimizes text compression performance, and ensures the stability and efficiency of data transmission.

Description

Technical Field

[0001] This invention relates to the field of ultra-large text transmission technology, and in particular to an optimization method for ultra-large text transmission based on dynamic block compression and routing. Background Technology

[0002] With the explosive growth of data volume, especially in the field of big data, the amount of text data is increasing exponentially. The transmission of ultra-large text files is becoming increasingly critical in many scenarios, such as cross-data center data synchronization, large-scale data backup and recovery, and distributed text analysis tasks. However, traditional transmission methods face many challenges when dealing with massive amounts of text data, including low transmission efficiency, huge bandwidth consumption, difficulty in ensuring the reliability of data transmission, and storage pressure during transmission.

[0003] Existing methods employ fixed-block transmission mechanisms, such as the data transmission mechanism of HDFS (Hadoop Distributed File System). HDFS uses the concept of data blocks, dividing large files into fixed-size data blocks for storage and transmission. The size of the data block is typically 128MB or 256MB, and data reliability is ensured through multiple data replicas.

[0004] However, for datasets with a high proportion of small files, 128MB blocks can lead to a large amount of fragmented storage, which increases the burden of metadata management and reduces I / O performance due to frequent cross-block operations during data reading. On the other hand, for very large files, fixed blocks cannot be flexibly adjusted according to the characteristics of the data, such as its popularity or access frequency, making it difficult to achieve efficient resource utilization.

[0005] The static compression strategy employed is to use a single compression algorithm globally to compress text files before transmission, thereby reducing file size and improving transmission efficiency. Commonly used compression algorithms include gzip, bzip2, and LZ4. These algorithms compress data by finding repeating patterns and redundant information in the text, thus reducing the amount of data transmitted.

[0006] However, different types of data have varying degrees of adaptability to compression algorithms. For example, for text data, dictionary-based compression algorithms may achieve higher compression ratios; while for binary data such as images and videos, the compression effects of LZ4 and Snappy are relatively limited. The global application of a single compression algorithm cannot be optimized for different data characteristics, which may prevent the full utilization of compression technology's advantages in some scenarios, resulting in wasted storage space and transmission bandwidth.

[0007] The shortest path routing method used: Existing systems typically use the Open Shortest Path First (OSPF) protocol for routing. OSPF determines data transmission routes by calculating the shortest path between nodes in the network topology. This method effectively ensures the efficiency and stability of data transmission in environments with relatively stable network topologies.

[0008] However, in practical applications, network link bandwidth utilization, latency, packet loss rate, and other conditions change dynamically over time. For example, during peak network periods, the shortest path may experience a significant increase in transmission latency or even data loss due to traffic congestion. OSPF-based shortest path routing cannot dynamically adjust its routing strategy based on these real-time network conditions, thus affecting the reliability and efficiency of data transmission and failing to meet the needs of application scenarios with high network quality requirements. Summary of the Invention

[0009] Based on the above analysis, this invention aims to disclose an optimization method for ultra-large text transmission based on dynamic block compression and routing; it aims to solve three core problems in the transmission of ultra-large text (≥1TB) in a distributed environment: block efficiency: how to dynamically block according to text semantics; compression bottleneck: dynamic selection of compression algorithms for multi-source heterogeneous text; routing oscillation: real-time network status perception and path decision.

[0010] This invention discloses an optimization method for ultra-large text transmission based on dynamic block compression and routing, comprising:

[0011] Step S101, Dynamic Blocking Step: Semantic relationship graph is constructed by semantic analysis of the ultra-large text. The ultra-large text is divided into logical paragraphs according to semantic relevance. The size of the block data is dynamically adjusted according to the access frequency of the logical paragraphs.

[0012] Step S102, Dynamic Compression Step: Based on the data characteristics of each data block, match the optimal compression algorithm for the data block and compress the data block.

[0013] Step S103, Dynamic Transmission Step: Based on the real-time monitoring of the current network status, dynamically predict the network congestion status. If there is no congestion, transmit the compressed data blocks to the target node one by one according to the original path; if there is congestion, replan the optimal path, update the SDN route, and continue to transmit the compressed data blocks to the target node according to the updated path.

[0014] Furthermore, the dynamic block division step includes:

[0015] Step S201: Perform semantic analysis on the massive text; In the semantic analysis, entities including names, organizations, and terms in the text are identified by named entity recognition, and the topic distribution of the text is extracted using the LDA topic model to help determine the semantic relationship between paragraphs.

[0016] Step S202: Based on the semantic analysis results, organize entities and topics into a semantic relationship diagram to represent the logical relationships between paragraphs;

[0017] Step S203: Divide the text into logical paragraphs based on the semantic map to ensure the semantic integrity of each paragraph;

[0018] Step S204: Traverse each logical segment and determine the granularity of the segmentation based on the access frequency; if the access frequency is high, the logical segment is cut into small-granularity blocks; if the access frequency is low, the logical segments are merged to obtain large-granularity blocks.

[0019] Furthermore, the dynamic compression step includes:

[0020] Step S301: Construct data feature parameters extracted from the block data and compression parameters to evaluate the compression effect of the compression algorithm;

[0021] Step S302: Construct a compression algorithm evaluation model that scores the compression algorithm by combining data feature parameters and compression parameters; this model is used to evaluate the compression effect of the compression algorithm.

[0022] Step S303: Traverse the candidate compression algorithms in the compression algorithm pool to compress the input block text, and use the compression algorithm evaluation model to evaluate the optimal compression algorithm for compressing the block text.

[0023] Step S304: Compress the block data using the optimal compression algorithm to obtain compressed data.

[0024] Furthermore, the data feature parameters extracted from the segmented data include data type, redundancy, entropy value, and access frequency;

[0025] Compression parameters used to evaluate the compression performance of a compression algorithm include compression ratio, compression time, and decompression time.

[0026] Furthermore, the evaluation model for scoring compression algorithms is as follows:

[0027] score=(1-H)*R*W r *ratio+F*W d *(1 / d_time)+(1-F)*W c *(1 / c_time)

[0028] Where score is the numerical value, H is the entropy value, R is the repetition rate, F is the access frequency; ratio is the compression ratio, c_time is the compression time, d_time is the decompression time, and W... r W c and W d These are the weight parameters.

[0029] Furthermore, before dynamic compression, the weight parameters are initialized:

[0030] During dynamic compression, the weight parameters are adjusted according to the data type of the data blocks.

[0031] When the data type is text, increase the weight W corresponding to the compression ratio. r When the data type is binary, increase the weight W corresponding to the decompression time d_time. d .

[0032] Furthermore, the dynamic transmission steps include:

[0033] Step S401: Start transmission based on the data block to be transmitted and the target node of transmission, deploy detection probes, and collect key performance indicators of the current network;

[0034] Step S402: Predict the network congestion probability based on the collected current key network performance indicators;

[0035] Step S403: Determine if there is congestion based on the congestion probability; if not, maintain the original path and proceed to the next step; if yes, recalculate the optimal path and proceed to the next step.

[0036] Step S404: Input the path into the SDN controller to perform route update;

[0037] Step S405: Transmit the data block to the target node according to the updated path.

[0038] Furthermore, key performance indicators of the current network include: bandwidth utilization, latency, and packet loss rate.

[0039] Furthermore, a trained long short-term memory network model is used to predict the network congestion probability; and whether congestion occurs is determined based on whether the predicted network congestion probability is greater than a preset congestion threshold.

[0040] Furthermore, the process of recalculating the optimal path and updating the SDN routes includes:

[0041] 1) Construct the topology of the current network;

[0042] 2) Use Dijkstra's algorithm to calculate the optimal path from the current node to the target node;

[0043] 3) Calculate path weights dynamically based on current network metrics to reflect the actual network status;

[0044] 4) Based on the topology, optimal path, and path weight, the SDN controller updates the network flow table and switches the data transmission path to the newly calculated optimal path.

[0045] The ultra-large text transmission optimization method based on dynamic block compression and routing disclosed in this invention can achieve at least one of the following beneficial effects:

[0046] Improving the efficiency of transmitting ultra-large text: By adopting a dynamic segmentation strategy based on semantic analysis, natural language processing technology is used to identify semantic units and logical paragraphs in the text for reasonable segmentation. Combined with data access history and prediction models, the segment size is dynamically adjusted. This allows for more efficient reading of frequently accessed hot content with smaller segment granularity, while using larger segments for low-frequency content to reduce metadata management costs, thereby improving the overall efficiency of transmitting ultra-large text.

[0047] Optimize text compression performance: The intelligent compression algorithm selection mechanism for multi-source heterogeneous text can extract features for different types of text data and dynamically select the optimal compression algorithm based on real-time evaluation results. It fully leverages the advantages of various compression algorithms to achieve comprehensive optimization of compression ratio, compression time, and decompression time, maximizing compression performance and effectively reducing the storage space and transmission bandwidth usage of ultra-large texts.

[0048] Ensuring the stability and efficiency of data transmission: A real-time network status awareness and dynamic routing decision-making scheme is implemented. By deploying network monitoring probes to collect key indicators in real time and using machine learning algorithms to predict network status, the scheme re-plans transmission paths when link congestion trends are observed, and introduces software-defined networking technology to achieve centralized management and flexible scheduling of network traffic, avoids routing oscillations, and ensures that ultra-large text data can be transmitted stably and efficiently in a distributed environment. Attached Figure Description

[0049] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0050] Figure 1 This is a flowchart of the ultra-large text transmission optimization method in an embodiment of the present invention;

[0051] Figure 2 This is a flowchart of the dynamic block division steps in an embodiment of the present invention;

[0052] Figure 3 This is a flowchart of the dynamic compression steps in an embodiment of the present invention;

[0053] Figure 4 This is a flowchart of the dynamic transmission steps in an embodiment of the present invention. Detailed Implementation

[0054] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and, together with the embodiments of the present invention, serve to illustrate the principles of the present invention.

[0055] One embodiment of the present invention discloses an optimization method for ultra-large text transmission based on dynamic block compression and routing, such as... Figure 1 As shown, it includes:

[0056] Step S101, Dynamic Blocking Step: Semantic relationship graph is constructed by semantic analysis of the ultra-large text. The ultra-large text is divided into logical paragraphs according to semantic relevance. The size of the block data is dynamically adjusted according to the access frequency of the logical paragraphs.

[0057] Step S102, Dynamic Compression Step: Based on the data characteristics of each data block, match the optimal compression algorithm for the data block and compress the data block.

[0058] Step S103, Dynamic Transmission Step: Based on the real-time monitoring of the current network status, dynamically predict the network congestion status. If there is no congestion, transmit the compressed data blocks to the target node one by one according to the original path; if there is congestion, replan the optimal path, update the SDN route, and continue to transmit the compressed data blocks to the target node according to the updated path.

[0059] Specifically, the dynamic block division step, such as Figure 2 The following are included:

[0060] Step S201: Perform semantic analysis on the massive text; In the semantic analysis, entities including names, organizations, and terms in the text are identified by named entity recognition, and the topic distribution of the text is extracted using the LDA topic model to help determine the semantic relationship between paragraphs.

[0061] Step S202: Based on the semantic analysis results, organize entities and topics into a semantic relationship diagram to represent the logical relationships between paragraphs;

[0062] Step S203: Divide the text into logical paragraphs based on the semantic map to ensure the semantic integrity of each paragraph;

[0063] Step S204: Traverse each logical segment and determine the granularity of the segmentation based on the access frequency; if the access frequency is high, the logical segment is divided into small-granularity blocks; if the access frequency is low, the logical segments are merged to obtain large-granularity blocks.

[0064] Specifically, step S204 includes:

[0065] 1) Determine the access frequency of logical paragraphs based on their access history;

[0066] 2) Determine if the access frequency exceeds the set popularity threshold; if yes, divide the logical segment into data blocks according to the set granularity; otherwise, proceed to the next step.

[0067] 3) Determine whether logical paragraphs can be merged; if yes, merge the logical paragraphs to obtain large-granularity block data; if no, treat the logical paragraphs as block data.

[0068] The block partitioning steps employed in this embodiment address the problem that existing Hadoop HDFS fixed-block partitioning methods cannot perceive data characteristics, resulting in low efficiency for transmitting extremely large text files. By introducing Natural Language Processing (NLP) technology, semantic analysis is performed on the text. Through named entity recognition and techniques such as LDA topic modeling, semantic units and logical paragraphs in the text are identified, and semantically related text content is divided into the same data block.

[0069] For example, when processing academic paper collections, papers on the same topic and their references can be grouped together to avoid mixing irrelevant content. Simultaneously, by combining data access history and predictive models, the size of the blocks can be dynamically adjusted. For frequently accessed, hot content, the block granularity can be appropriately reduced to improve reading efficiency and speed up access; while low-frequency content can be merged into larger blocks to reduce storage space waste and lower metadata management costs. This dynamic block-splitting method not only improves the flexibility and efficiency of text processing but also adapts to different application scenarios and datasets through adaptive adjustment strategies, providing a more meaningful foundation for subsequent text analysis and processing.

[0070] Specifically, the dynamic compression steps, such as Figure 3 The following are included:

[0071] Step S301: Construct data feature parameters extracted from the block data and compression parameters to evaluate the compression effect of the compression algorithm;

[0072] Step S302: Construct a compression algorithm evaluation model that scores the compression algorithm by combining data feature parameters and compression parameters; this model is used to evaluate the compression effect of the compression algorithm.

[0073] Step S303: Traverse the candidate compression algorithms in the compression algorithm pool to compress the input block text, and use the compression algorithm evaluation model to evaluate the optimal compression algorithm for compressing the block text.

[0074] Step S304: Compress the block data using the optimal compression algorithm to obtain compressed data.

[0075] Specifically, in step S301, the data feature parameters extracted from the segmented data include data type, repetition degree, entropy value, and access frequency; wherein,

[0076] Data type indicates whether the chunked data is structured or unstructured; redundancy and entropy values ​​indicate the content characteristics of the chunked data; access frequency indicates whether the chunked data needs to be frequently decompressed in subsequent applications.

[0077] Compression parameters used to evaluate the compression performance of a compression algorithm include compression ratio, compression time, and decompression time.

[0078] Specifically, in step S302, the compression algorithm evaluation model for scoring the compression algorithm is as follows:

[0079] score=(1-H)*R*W r *ratio+F*W d *(1 / d_time)+(1-F)*W c *(1 / c_time)

[0080] Where score is the numerical value, H is the entropy value, R is the repetition rate, F is the access frequency; ratio is the compression ratio, c_time is the compression time, d_time is the decompression time, and W... r W c and W d These are the weight parameters.

[0081] Preferably, before dynamic compression, the weight parameter W is initialized. r It is 0.5; W c W is 0.2. d It is 0.3;

[0082] During dynamic compression, the weight parameters are adjusted according to the data type of the data blocks; when the data type is text, the weight W corresponding to the compression ratio is increased. r The value is 1.2; when the data type is binary, the weight W corresponding to the decompression time is increased. d Fear 1.5.

[0083] Specifically, in step S303, compression algorithms including "Zstandard", "LZ4", "Gzip" and "BZIP2" are traversed.

[0084] Based on the scoring, the Zstandard compression algorithm is selected as the preferred compression algorithm for code-type texts, as it is based on dictionary compression and is the preferred compression algorithm for code-type texts with high repetition. For news and information-type texts, considering timeliness, the LZ4 compression algorithm is selected as the preferred compression algorithm, as it has a fast compression speed and is the preferred compression algorithm for news and information-type texts with high timeliness requirements.

[0085] By establishing a compression algorithm evaluation model, the compression ratio, compression time, and decompression time of different compression algorithms under the current data characteristics are tested in real time. Based on the evaluation results, the optimal compression algorithm is dynamically selected to maximize compression performance.

[0086] Specifically, the dynamic transmission steps, such as Figure 4 The following are included:

[0087] Step S401: Start transmission based on the data block to be transmitted and the target node of transmission, deploy detection probes, and collect key performance indicators of the current network;

[0088] Step S402: Predict the network congestion probability based on the collected current key network performance indicators;

[0089] Step S403: Determine if there is congestion based on the congestion probability; if not, maintain the original path and proceed to the next step; if yes, recalculate the optimal path and proceed to the next step.

[0090] Step S404: Input the path into the SDN controller to perform route update;

[0091] Step S405: Transmit the data block to the target node according to the updated path.

[0092] Specifically, in step S401, the key performance indicators of the current network are collected, including bandwidth utilization, latency, and packet loss rate.

[0093] Specifically, in step S402, a trained long short-term memory network model is used to predict the network congestion probability; and whether congestion occurs is determined based on whether the predicted network congestion probability is greater than a preset congestion threshold.

[0094] Specifically, in step S404, the process of inputting the path to the SDN controller for route update includes:

[0095] 1) Construct the topology of the current network;

[0096] 2) Use Dijkstra's algorithm to calculate the optimal path from the current node to the target node;

[0097] 3) Calculate path weights dynamically based on current network metrics to reflect the actual network status;

[0098] 4) Based on the topology, optimal path, and path weight, the SDN controller updates the network flow table and switches the data transmission path to the newly calculated optimal path.

[0099] The dynamic transmission steps employed in this embodiment address the issue of OSPF-based shortest path routing's inability to adapt to changing network conditions. By deploying network monitoring probes, key indicators such as bandwidth utilization, latency, and packet loss rate of each link are collected in real time. Machine learning algorithms, including Long Short-Term Memory (LSTM) networks, are used to predict network conditions. When congestion is detected on a link, routing calculation methods such as Dijkstra's algorithm are combined to replan the data transmission path, selecting the path with the optimal overall performance under the current network conditions. Simultaneously, Software-Defined Networking (SDN) technology is introduced to separate the network control plane from the data plane, enabling centralized management and flexible scheduling of network traffic. This effectively avoids routing oscillations and ensures stable and efficient transmission of massive text data in a distributed environment.

[0100] In summary, the ultra-large text transmission optimization method based on dynamic block compression and routing in this embodiment can achieve the following:

[0101] 1. Improve the efficiency of transmitting ultra-large texts: By adopting a dynamic segmentation strategy based on semantic analysis, natural language processing technology is used to identify semantic units and logical paragraphs in the text for reasonable segmentation. Combined with data access history and prediction models, the segment size is dynamically adjusted. This allows for more efficient reading of frequently accessed hot content with smaller segment granularity, while using larger segments for low-frequency content to reduce metadata management costs, thereby improving the overall efficiency of transmitting ultra-large texts.

[0102] 2. Optimize text compression performance: The intelligent compression algorithm selection mechanism for multi-source heterogeneous text can extract features for different types of text data and dynamically select the optimal compression algorithm based on real-time evaluation results. It fully leverages the advantages of various compression algorithms to achieve comprehensive optimization of compression ratio, compression time, and decompression time, maximizing compression performance and effectively reducing the storage space and transmission bandwidth usage of ultra-large texts.

[0103] 3. Ensure the stability and efficiency of data transmission: The real-time network status perception and dynamic routing decision-making scheme collects key indicators in real time by deploying network monitoring probes and uses machine learning algorithms to predict network status. When link congestion trends occur, the transmission path is replanned by combining routing calculation methods. At the same time, software-defined networking technology is introduced to achieve centralized management and flexible scheduling of network traffic, avoid routing oscillations, and ensure that ultra-large text data can be transmitted stably and efficiently in a distributed environment.

[0104] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for optimizing ultra-large text transmission based on dynamic block compression and routing, characterized in that, include: Step S101, Dynamic Blocking Step: Semantic relationship graph is constructed by semantic analysis of the ultra-large text. The ultra-large text is divided into logical paragraphs according to semantic relevance. The size of the block data is dynamically adjusted according to the access frequency of the logical paragraphs. Based on the access history of logical paragraphs, determine the access frequency of logical paragraphs; determine whether the access frequency exceeds the set popularity threshold. If so, the logical paragraphs will be divided into data blocks according to the set granularity; If no, determine whether the logical paragraphs can be merged; if yes, merge the logical paragraphs to obtain large-granular block data. No, then the logical paragraphs will be treated as data blocks; Step S102, Dynamic compression step; Based on the data characteristics of each data block, the optimal compression algorithm is matched for the data block and the data block is compressed. Step S103, Dynamic transmission step: Based on the real-time monitoring of the current network status, the trained long short-term memory network model is used to predict the probability of network congestion; if there is no congestion, the compressed data blocks are transmitted to the target node one by one according to the original path. If congestion occurs, the optimal path is replanned, the SDN route is updated, and the compressed data blocks are transmitted to the target node according to the updated path. The dynamic compression process includes: Step S301: Construct data feature parameters extracted from the block data and compression parameters to evaluate the compression effect of the compression algorithm; The data feature parameters extracted from the segmented data include data type, repetition degree, entropy value, and access frequency; among them, Data type indicates whether the chunked data is structured or unstructured; redundancy and entropy values ​​indicate the content characteristics of the chunked data; access frequency indicates whether the chunked data needs to be frequently decompressed in subsequent applications. Step S302: Construct a compression algorithm evaluation model that scores the compression algorithm by combining data feature parameters and compression parameters; this model is used to evaluate the compression effect of the compression algorithm. The evaluation model for scoring compression algorithms is as follows: in, For fractions, R is the entropy value, and R is the repetition rate. F To increase visitor popularity; For compression ratio, To compress time ， The decompression process takes time. , and These are weight parameters; Step S303: Traverse the candidate compression algorithms in the compression algorithm pool to compress the input block text, and use the compression algorithm evaluation model to evaluate the optimal compression algorithm for compressing the block text. Step S304: Compress the block data using the optimal compression algorithm to obtain compressed data; Initialize the weight parameters before dynamic compression; During dynamic compression, the weight parameters are adjusted according to the data type of the data blocks. Increase the compression ratio when the data type is text. Corresponding weights When the data type is binary, the decompression time will increase. Corresponding weights .

2. The method for optimizing ultra-large text transmission based on dynamic block compression and routing according to claim 1, characterized in that, The dynamic block partitioning process includes: Step S201: Perform semantic analysis on the massive text; In the semantic analysis, entities including names, organizations, and terms in the text are identified by named entity recognition, and the topic distribution of the text is extracted using the LDA topic model to help determine the semantic relationships between paragraphs; Step S202: Based on the semantic analysis results, organize entities and topics into a semantic relationship diagram to represent the logical relationships between paragraphs; Step S203: Divide the text into logical paragraphs based on the semantic map to ensure the semantic integrity of each paragraph; Step S204: Traverse each logical segment and determine the granularity of the segmentation based on the access frequency; if the access frequency is high, the logical segment is cut into small-granularity blocks; if the access frequency is low, the logical segments are merged to obtain large-granularity blocks.

3. The method for optimizing ultra-large text transmission based on dynamic block compression and routing according to claim 1, characterized in that, The dynamic transmission steps include: Step S401: Start transmission based on the data block to be transmitted and the target node of transmission, deploy detection probes, and collect key performance indicators of the current network; Step S402: Predict the network congestion probability based on the collected current key network performance indicators; Step S403: Determine if there is congestion based on the congestion probability; if not, maintain the original path and proceed to the next step; if yes, recalculate the optimal path and proceed to the next step. Step S404: Input the path into the SDN controller to perform route update; Step S405: Transmit the data block to the target node according to the updated path.

4. The method for optimizing ultra-large text transmission based on dynamic block compression and routing according to claim 3, characterized in that, Key performance metrics collected for the current network include: bandwidth utilization, latency, and packet loss rate.

5. The method for optimizing ultra-large text transmission based on dynamic block compression and routing according to claim 3, characterized in that, A trained long short-term memory network model is used to predict the probability of network congestion; and whether congestion occurs is determined based on whether the predicted network congestion probability is greater than a preset congestion threshold.

6. The method for optimizing ultra-large text transmission based on dynamic block compression and routing according to claim 3, characterized in that, The process of recalculating the optimal path and updating SDN routes includes: 1) Construct the topology of the current network; 2) Use Dijkstra's algorithm to calculate the optimal path from the current node to the target node; 3) Dynamically calculate path weights based on current network metrics to reflect the actual network status; 4) Based on the topology, optimal path, and path weight, the SDN controller updates the network flow table and switches the data transmission path to the newly calculated optimal path.

Images