Intelligent data compression method and system based on multi-protocol heterogeneous device interconnection

Abstract

The invention relates to the technical field of data compression, in particular to an intelligent data compression method and system based on multi-protocol heterogeneous equipment interconnection. The method comprises the following steps: when it is detected that a new heterogeneous device applies to join a network, carrying out communication protocol real-time analysis and protocol conversion adaptation to obtain a heterogeneous device adaptation protocol; carrying out multi-device data stream receiving according to a heterogeneous device adaptation protocol, and generating data streams of a plurality of time windows; adaptive coding and multi-level compression are carried out on the data streams of the multiple time windows, and a multi-device adaptive compression mechanism is constructed; identifying compression load characteristics of each heterogeneous device, and performing dynamic task allocation to obtain an intelligent task allocation strategy; and driving real-time data intelligent compression operation based on an intelligent task allocation strategy and a multi-device adaptive compression mechanism. According to the invention, efficient and adaptive data compression of different devices is realized, and the data transmission efficiency and quality are improved.

Description

Technical Field

[0001] This invention relates to the field of data compression technology, and in particular to a data intelligent compression method and system based on the interconnection of multi-protocol heterogeneous devices. Background Technology

[0002] In the operation of interconnected heterogeneous devices using multiple protocols, the different communication protocols, data formats, sampling frequencies, and transmission mechanisms employed by each device lead to problems such as inconsistent formats, high redundancy, and high bandwidth consumption during data aggregation, transmission, and storage. Simultaneously, as the number of devices and the amount of data continue to increase, network transmission links are prone to congestion, increasing data processing latency and impacting the overall system efficiency. Furthermore, some devices generate data with significant time-varying and non-linear characteristics, making traditional fixed compression algorithms ill-suited to this complex and dynamic scenario. This results in low compression ratios, poor real-time performance, and may even affect the integrity and stability of data transmission. These problems not only increase the burden on the network and storage but also create potential performance bottlenecks for subsequent data processing and analysis.

[0003] Traditional data compression methods are mostly based on a single protocol and fixed data structure, relying primarily on static algorithms and predefined rules. While these methods can achieve basic data compression in specific scenarios, they often lack the ability to identify multi-source heterogeneous data and cannot flexibly adjust compression strategies according to dynamic changes in protocol characteristics and data types. Furthermore, most existing compression methods require manual configuration or periodic maintenance, making it difficult to achieve efficient real-time compression and transmission optimization. Their overall level of intelligence and automation is low, failing to meet the demands of modern multi-protocol heterogeneous networks for efficient data processing. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention proposes a data intelligent compression method and system based on the interconnection of heterogeneous devices with multiple protocols, thereby solving at least one of the aforementioned technical problems.

[0005] To achieve the above objectives, the present invention provides a data intelligent compression method based on the interconnection of multi-protocol heterogeneous devices, comprising the following steps: Step S1: When a new heterogeneous device is detected requesting to join the network, real-time parsing and protocol conversion and adaptation are performed to obtain the heterogeneous device adaptation protocol; Step S2: Receive data streams from multiple devices according to the heterogeneous device adaptation protocol, and generate data streams for multiple time windows; Step S3: Perform adaptive encoding and multi-level compression on data streams from multiple time windows to build a multi-device adaptive compression mechanism; Step S4: Identify the compression load characteristics of each heterogeneous device, perform dynamic task allocation, and thus obtain an intelligent task allocation strategy; Step S5: Drive real-time intelligent data compression operations based on intelligent task allocation strategies and multi-device adaptive compression mechanisms.

[0006] This specification provides a data intelligent compression system based on the interconnection of multi-protocol heterogeneous devices, used to execute the data intelligent compression method based on the interconnection of multi-protocol heterogeneous devices as described above, including: The protocol parsing module is used to perform real-time parsing and protocol conversion and adaptation when a new heterogeneous device is detected requesting to join the network, so as to obtain the heterogeneous device adaptation protocol. The data stream segmentation module is used to receive data streams from multiple devices according to the heterogeneous device adaptation protocol and generate data streams with multiple time windows. The multi-level compression module is used to adaptively encode and compress data streams from multiple time windows, and to build a multi-device adaptive compression mechanism. The intelligent task allocation module is used to identify the compression load characteristics of each heterogeneous device, perform dynamic task allocation, and thus obtain an intelligent task allocation strategy. The collaborative control module is used to drive real-time intelligent data compression operations based on intelligent task allocation strategies and multi-device adaptive compression mechanisms.

[0007] The specific benefits of this invention are as follows: Through real-time parsing and adaptation, protocol interoperability between different devices can be achieved, enabling devices using various communication protocols (such as MQTT, CoAP, Modbus, Zigbee, etc.) to uniformly access the network. When a new device is connected, there is no need to redevelop the entire communication mechanism; only protocol adaptation is required, significantly improving the system's flexibility and adaptability. It eliminates the need for manual configuration of communication protocols, reducing system maintenance costs and access time. Data streams from different devices are uniformly converted into time-window format, facilitating subsequent parallel processing, compression, and analysis. Based on time-window segmentation, data aggregation and optimization along the time dimension can be achieved, facilitating real-time processing and historical analysis. The time-window mechanism solves the problem of time-series misalignment in asynchronously reported data during the analysis phase. The optimal encoding and compression algorithm is dynamically selected based on device data characteristics, achieving a higher compression ratio. It effectively reduces the amount of raw data, lowering storage pressure on edge nodes and transmission burden on the cloud. The multi-level compression mechanism (such as edge pre-compression + central further compression) is suitable for device environments with limited computing power or bandwidth. Task allocation is dynamically adjusted based on the current status of devices (such as CPU load, memory usage, network bandwidth, etc.) to avoid overloading some devices. Intelligent task scheduling matches compression tasks with optimal computing resources, improving the overall system's parallel processing capabilities and response speed. When a device malfunctions or experiences excessive load, tasks can be promptly migrated to other devices, enhancing system stability and reliability. Combining protocol adaptation, data integration, compression optimization, and task scheduling strategies from the previous steps forms a highly efficient and collaborative compression system. This ensures stable and efficient data compression even under multi-source data and high-concurrency scenarios. Moving compression computation to the edge improves system response speed, reduces dependence on central servers, and enables real-time decision-making and intelligent processing. Attached Figure Description

[0008] Figure 1 This is a flowchart illustrating the steps of an intelligent data compression method based on the interconnection of multi-protocol heterogeneous devices according to the present invention. Figure 2 This is a detailed flowchart illustrating the implementation steps of step S1. Figure 3 This is a flowchart illustrating the detailed implementation steps of step S2. Detailed Implementation

[0009] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0010] This application provides a data intelligent compression method and system based on the interconnection of heterogeneous devices using multiple protocols. The executing entities of the data intelligent compression method and system based on the interconnection of heterogeneous devices using multiple protocols include, but are not limited to, mechanical equipment, data processing platforms, cloud server nodes, network upload devices, etc., which can be considered as general computing nodes in this application. The data processing platform includes, but is not limited to, at least one of an audio / image management system, an information management system, and a cloud-based data management system.

[0011] Please see Figures 1 to 3 This invention provides a data intelligent compression method based on the interconnection of heterogeneous devices using multiple protocols, comprising the following steps: Step S1: When a new heterogeneous device is detected requesting to join the network, real-time parsing and protocol conversion and adaptation are performed to obtain the heterogeneous device adaptation protocol; Step S2: Receive data streams from multiple devices according to the heterogeneous device adaptation protocol, and generate data streams for multiple time windows; Step S3: Perform adaptive encoding and multi-level compression on data streams from multiple time windows to build a multi-device adaptive compression mechanism; Step S4: Identify the compression load characteristics of each heterogeneous device, perform dynamic task allocation, and thus obtain an intelligent task allocation strategy; Step S5: Drive real-time intelligent data compression operations based on intelligent task allocation strategies and multi-device adaptive compression mechanisms.

[0012] In the embodiments of the present invention, see Figure 1 This is a flowchart illustrating the steps of a data intelligent compression method based on multi-protocol heterogeneous device interconnection according to the present invention. In this example, the steps of the data intelligent compression method based on multi-protocol heterogeneous device interconnection include: Step S1: When a new heterogeneous device is detected requesting to join the network, real-time parsing and protocol conversion and adaptation are performed to obtain the heterogeneous device adaptation protocol; In this embodiment, when a new heterogeneous device requests to join the network, the protocol detection engine is first activated. Deep packet inspection (DPI) technology is used to analyze the initial handshake data packets sent by the device, identifying its protocol signature and version information. A protocol identification library containing templates for mainstream protocols is maintained, covering 15 common protocols such as KNX, Zigbee, LoRa, Modbus, and BACnet. Each protocol template includes key information such as header structure, data field definitions, and verification algorithms. The protocol identification process employs a multi-level matching strategy. First, header feature matching is performed, achieving an accuracy of 98.5% with an identification time controlled within 20 milliseconds. When an unknown protocol is detected, a protocol learning mechanism is automatically triggered, analyzing the structural patterns of 100 continuously sampled data packets to establish a temporary protocol template. The protocol conversion and adaptation module uses a Standardized Data Container (SDC) as an intermediate format, mapping the original protocol data to a unified JSON structure containing standard fields such as device identifier, data type, numerical content, timestamp, and quality identifier. During the conversion process, a data cleaning mechanism is implemented to filter out erroneous frames that fail CRC checks and eliminate redundant data transmitted repeatedly. The data cleaning rate is typically maintained at 5-8%. Establish a protocol conversion performance monitoring system to collect real-time statistics on conversion latency, success rate, error types, and other indicators. When the conversion latency exceeds 50 milliseconds, an optimization mechanism will be automatically triggered to improve conversion efficiency by caching commonly used conversion rules and parallel processing.

[0013] Step S2: Receive data streams from multiple devices according to the heterogeneous device adaptation protocol, and generate data streams for multiple time windows; In this embodiment, a data receiving pipeline established based on a heterogeneous device adaptation protocol employs a sliding time window technique to segment and process data streams from multiple devices. The time window size is dynamically adjusted according to device type and data update frequency: a 5-second time window is used for high-frequency sensor devices, a 15-second time window for medium-frequency control devices, and a 60-second time window for low-frequency status devices. Data within each time window is grouped by device ID, and a device-timestamp index table is established to support fast querying and statistical analysis. A real-time data stream feature extraction mechanism is implemented, calculating statistical features for the data within each time window, including 12 key indicators such as data arrival frequency, numerical variation amplitude, data type distribution, and outlier ratio. The data stream cache adopts a circular buffer design with a buffer size of 16MB, capable of holding approximately 1 million standard data records. When the buffer utilization exceeds 80%, a data compression and dumping mechanism is automatically initiated. A 50% overlap rate is used for the time window overlap strategy to ensure the accuracy of data continuity analysis and avoid the impact of time boundary effects on data feature extraction. A data stream quality assessment mechanism is established to score the data integrity, consistency, and timeliness of each time window. Data in time windows with a quality score below 0.7 will be marked as abnormal and subject to special handling. A multi-device data stream synchronization mechanism ensures the time consistency of data across devices through a clock synchronization protocol, with time synchronization accuracy controlled within 10 milliseconds.

[0014] Step S3: Perform adaptive encoding and multi-level compression on data streams from multiple time windows to build a multi-device adaptive compression mechanism; In this embodiment, a three-layer adaptive encoding and compression architecture is constructed to address the characteristic differences of different data types. The first layer is protocol layer compression, which utilizes the inherent structural characteristics of protocol messages for template matching compression, achieving a compression ratio of 60-75% for standard KNX messages. Protocol layer compression employs prefix encoding technology, encoding common protocol header fields into short bit sequences, such as compressing a 16-byte complete KNX message header into a 4-byte identifier code. The second layer is data content compression, selecting the optimal encoding algorithm based on the data type. Numerical data uses improved differential encoding, encoding and transmitting continuous numerical changes; when the temperature sensor data change is less than 0.1 degrees, only a 1-byte identifier is transmitted. Boolean switch state data uses run-length encoding, merging consecutive identical states; when the continuous holding time of a switch state exceeds 30 seconds, RLE compression is enabled, achieving a compression ratio of 70-85%. The third layer is cross-device associated compression. It establishes a joint encoding mode by analyzing the correlation between data from different devices. When the correlation between the data changes from the temperature and humidity sensor and the air conditioner controller exceeds 0.8, a joint compression dictionary is established, transmitting only the master device data and correlation coefficients. Data from the slave devices is recovered through correlation calculations. The adaptive encoding selection mechanism dynamically adjusts the encoding strategy based on real-time compression effect feedback, maintaining an encoding effect evaluation table that records the compression ratio and computational cost of each encoding algorithm under different data types. If the compression ratio of a certain encoding method falls below the expected value three times consecutively, it automatically switches to an alternative algorithm. The multi-device adaptive compression mechanism also includes dynamic compression quality adjustment, balancing compression ratio and compression speed according to network bandwidth and latency requirements. High-priority data uses fast compression algorithms to ensure low latency, while ordinary data uses high-compression-ratio algorithms to save bandwidth.

[0015] Step S4: Identify the compression load characteristics of each heterogeneous device, perform dynamic task allocation, and thus obtain an intelligent task allocation strategy; In this embodiment, the compression processing capabilities and load characteristics of each device are identified by continuously monitoring the hardware performance indicators and operating status of heterogeneous devices. The device performance profile includes real-time monitoring data across eight dimensions, such as CPU utilization, memory usage, storage space, and network bandwidth, and is updated every 30 seconds. A compression capability evaluation model is established based on the historical compression task execution data of each device. High-performance edge computing devices are rated 8-10 points for compression processing capability, medium-performance gateway devices are rated 5-7 points, and resource-constrained sensor devices are rated 1-3 points. A compression load characteristic identification algorithm analyzes the computational complexity of different compression tasks. Simple header compression tasks are rated 1-2 points, data content compression tasks are rated 3-5 points, and cross-device associated compression tasks are rated 6-8 points. The intelligent task allocation strategy employs a load balancing algorithm, prioritizing the allocation of high-complexity compression tasks to high-performance devices and simple tasks to resource-constrained devices, ensuring a balanced compression load across all devices. The task allocation decision considers communication latency and data transmission costs between devices, prioritizing devices with network latency of less than 50 milliseconds and sufficient bandwidth to execute compression tasks. A dynamic task migration mechanism is established. When the compression load on a device exceeds 90%, some tasks are automatically migrated to other idle devices, with the migration latency controlled within 100 milliseconds. The intelligent task allocation strategy also includes a fault-tolerance mechanism. When a device fails or goes offline, its compression tasks are automatically reassigned to a backup device within 5 seconds, ensuring the continuity and reliability of the compression service. A task allocation performance evaluation mechanism regularly tracks metrics such as task completion rate, average processing time, and resource utilization for each device, and continuously optimizes the task allocation algorithm based on the evaluation results.

[0016] Step S5: Drive real-time intelligent data compression operations based on intelligent task allocation strategies and multi-device adaptive compression mechanisms.

[0017] In this embodiment, a real-time data intelligent compression job scheduling engine is activated based on an intelligent task allocation strategy and a multi-device adaptive compression mechanism. Compression job scheduling employs a priority queue mechanism, with emergency control command data having the highest priority (P1), real-time sensor data having medium priority (P2), and historical log data having low priority (P3), ensuring timely processing of critical data. The job scheduling engine maintains a global compression task queue, monitoring the task execution status and queue length of each device in real time. When the task queue length of a device exceeds 100 tasks, a load redistribution mechanism is triggered. A quality control mechanism is implemented during compression job execution, setting an execution timeout for each compression task: 50 milliseconds for P1 tasks, 200 milliseconds for P2 tasks, and 1000 milliseconds for P3 tasks. Timed-out tasks are automatically marked as failed and rescheduled. A real-time feedback mechanism for compression effectiveness is established, statistically analyzing key indicators such as the average compression ratio, compression time, and decompression success rate for each data type. Tasks with a compression ratio below 50% are automatically switched to alternative compression algorithms. The intelligent compression operation also includes adaptive parameter adjustment, dynamically adjusting the compression intensity based on network congestion levels. When network bandwidth is sufficient, a fast compression algorithm is used to ensure low latency; when network congestion occurs, the compression ratio is increased to reduce data transmission volume. Job execution monitoring provides a visual interface displaying real-time information such as compression task execution status, device load distribution, and network transmission efficiency. Administrators can adjust compression parameters and strategy configurations through this interface. An anomaly handling and recovery mechanism is also established. When the compression failure rate exceeds 5%, a fault diagnosis program is automatically initiated to analyze the causes of failure and execute corresponding recovery strategies, ensuring the stable and reliable operation of the entire intelligent data compression process.

[0018] In this embodiment, see Figure 2 The diagram below illustrates the detailed implementation steps of step S1. In this embodiment, the detailed implementation steps of step S1 include: Real-time monitoring of network space status; when a new heterogeneous device requests to join the network, preload containerized parsing units from the cloud protocol library. The containerized parsing unit performs real-time parsing of the communication protocol of the heterogeneous device to extract the communication protocol information of the heterogeneous device. The communication protocol information includes the protocol header structure, data payload mode and timing features. The communication protocol information is analyzed for device type to obtain the device type identifier; Based on the communication protocol information, analyze the protocol stack hierarchy and data transmission frequency mode, and combine the device type identifier to perform device digital fingerprint evolution, and construct a heterogeneous device digital fingerprint map. Based on the digital fingerprint map of heterogeneous devices, protocol conversion and adaptation are performed in the network space to obtain the heterogeneous device adaptation protocol.

[0019] In this embodiment, a high-performance traffic acquisition probe is deployed at the network core layer to continuously monitor the data flow in the network with millisecond-level time precision through mirrored ports or distributed collection mechanisms. After entering the detection module, all traffic data first passes through a high-speed cache queue and is then processed by a fast identification algorithm based on hash index. For each access request, its source MAC address, source IP address, transport layer port information, and the five-tuple parameters of the initial communication session are extracted, and an index value is generated using a hash function. This index value is then compared with the existing device fingerprint table in O(1) complexity. When a mismatched index is found, it is immediately marked as a "new access device," and its communication data flow is completely recorded, including the first few bidirectional data packets and their high-precision timing information. The entire detection process adopts a multi-threaded asynchronous processing mechanism to ensure that fast identification can still be achieved under high concurrency conditions. This step also introduces a Bloom filter to deduplicatively screen a large number of historical records to avoid duplicate identification. Based on its initial communication characteristics, an index is performed in the cloud protocol library to retrieve the most matching containerized parsing unit. The cloud-based protocol library employs a layered management structure, with different types of communication protocols corresponding to independent container images. Each image integrates a deep parsing engine and feature extraction logic specific to its protocol. Utilizing information such as port number distribution, packet length statistics, and field distribution from initial traffic packets, a similarity matching algorithm filters candidate images, and a preset threshold determines whether they meet loading conditions. Once a target container is selected, it is quickly pulled to edge nodes via the container image repository interface and automatically deployed. After the container starts, it receives communication traffic from the corresponding device in real time and performs structured analysis using a deep protocol parsing module. The parsing process covers various aspects, including the distribution of protocol header fields, the length variation patterns of payload data, the temporal characteristics of message interactions, and protocol layering relationships. The extracted results are uniformly packaged into a standardized description format for easy subsequent processing.

[0020] The system accurately determines the device type through automated protocol information feature analysis. The parsed protocol header structure, data payload features, and timing information are formatted and matched against a protocol-device type mapping table maintained in a rule base. The rule base defines multi-layered mapping relationships, covering application layer features, transport layer features, and combination matching logic for specific key fields. If a perfectly matching feature combination is found in the rule base, the device type identifier can be directly determined. For communication modes that fail rule matching, they are input into a classification model based on multi-dimensional feature vectors for identification. This model uses a gradient boosting decision tree as its core algorithm, training the input multi-dimensional protocol information and then predicting the corresponding device type label. Input features include multiple dimensions such as the number of protocol fields, length variance, statistics of payload length distribution, spectral features of timing features, and session duration. This combination of rule matching and machine learning allows for accurate device type determination even when the communication protocol is unknown, providing device category information for subsequent fingerprint construction. After device type identification, the evolution and mapping of the device's digital fingerprint begins. The construction of digital fingerprints does not rely on a single static field, but rather integrates multi-dimensional information such as communication protocol stack structure, data transmission mode, and device type characteristics. The hierarchical structure of the protocol stack is meticulously divided, extracting structural features from the link layer, network layer, transport layer, and application layer, representing them as hierarchical feature vectors. Subsequently, feature extraction is performed on the time series of device communication using spectrum analysis algorithms, calculating peak frequency, bandwidth distribution, and periodic characteristics to form a time-series feature vector. Next, the device type identifier is combined with the aforementioned structural and time-series features through graph embedding operations, generating a fingerprint representation of uniform length using a node embedding algorithm. After fingerprint generation, its similarity is compared with existing device digital fingerprint maps. Locality-sensitive hashing (LSH) is used for fast searching to determine the fingerprint's position in the map. If it is an existing fingerprint type, its state information is updated; if it is a new type, it is added to the map as a new node and connected with similar nodes to form a new graph structure. The target device fingerprint is located in the map, and the nodes with the highest similarity are searched, analyzing the protocol stack characteristics and data transmission modes of these nodes. By comparing structural and temporal characteristics, the mapping relationship between the target device protocol and the target compatible protocol is determined. Subsequently, a protocol conversion strategy is automatically generated based on predefined mapping templates and field conversion rules. The conversion strategy covers various aspects, including data field mapping, packet format adjustment, and transmission frequency synchronization. After generating the strategy, corresponding protocol conversion containers are deployed in edge computing nodes, enabling real-time protocol translation through a streaming pipeline. The structural and temporal information contained in the digital fingerprint provides precise parameter support for the conversion process, allowing the conversion to go beyond simple field-level mapping and achieve adaptive adjustments at the data transmission frequency and mode levels.

[0021] In this embodiment, see Figure 3 The diagram below illustrates the detailed implementation steps of step S2. In this embodiment, the detailed implementation steps of step S2 include: Identify heterogeneous connected devices in cyberspace, perform device interaction analysis, and extract physical connection relationships, logical dependencies, and data flow paths between devices; Based on the physical connection relationship, logical dependency relationship and data flow path, the device topology association is parsed to obtain the heterogeneous device topology association structure; Based on the heterogeneous device topology and heterogeneous device adaptation protocol, multi-device data stream reception is performed, and data streams from different heterogeneous devices are received in advance. The data stream is divided into time-series windows to generate data streams with multiple time windows.

[0022] In this embodiment, a high-precision traffic acquisition module is deployed at the core network layer. Through mirror interfaces and distributed acquisition nodes, real-time capture of communication data across the entire network is performed and stored in a structured format as a five-tuple. The initial identifier of each device is composed of its physical address, IP address, port characteristics, and communication handshake information, forming the basic dataset for device identification. Based on this, an adjacency matrix construction algorithm is used to scan and correlate the communication behaviors between all devices. Physical connection identification is based on link layer and network layer information, focusing on analyzing the correspondence between MAC addresses and switch ports, ARP cache tables, and LLDP packet structures to determine the physical topology connections between devices and access nodes. Logical dependency analysis is based on the interaction characteristics of the transport layer and application layer, determining the service dependency chain between devices through request-response matching, session duration distribution, and connection direction. Data flow path extraction uses a directed graph traversal algorithm, abstracting the communication links between devices into directed edges and combining timestamp information to determine the data flow direction. The entire identification and interaction analysis process adopts a combined batch processing and streaming analysis mode to ensure high accuracy and real-time performance even in large-scale device scenarios. The identified device relationships are represented as a multi-layer graph structure, with the bottom layer being the physical connection layer, the middle layer being the logical dependency layer, and the top layer being the data flow layer. The physical layer establishes the basic network skeleton through the correspondence between nodes and links; the logical layer describes the service dependencies between devices through application interaction relationships; and the data flow layer represents the communication flow between devices in the form of time-ordered directed edges. A multi-layer graph fusion algorithm is used to uniformly encode the three-layer structure, transforming the multiple relationships between nodes into an adjacency tensor representation. Subsequently, through depth graph traversal and strongly connected component analysis, key nodes, boundary nodes, and clusters in the topology are identified, forming a topology association structure that can be used for subsequent processing. For complex heterogeneous protocol devices, a protocol adaptation identifier is introduced to map nodes of different protocol devices to a unified abstraction layer in the topology, thereby avoiding the impact of protocol differences on topology associations.

[0023] At each node of the topology, a corresponding protocol adaptation access unit is established. Containerized protocol adaptation modules are deployed at edge nodes to uniformly format data streams from different devices. Based on the previously established protocol conversion strategy, the adaptation module translates data in different protocol formats into a standardized intermediate representation in real time and binds it to the flow direction according to the topology. A centralized data flow scheduling mechanism is adopted to aggregate the output streams of all devices in the topology according to the connection relationships between nodes, and a message queue middleware is used to distribute the data streams to the receiving modules according to the topology path. During the receiving phase, data is classified using flow identifiers, timestamps, and topology layer information to ensure that stream data from different heterogeneous devices can be accurately distinguished. For device data with protocol differences, protocol layer conversion and identifier mapping are completed during reception to ensure that the data can be uniformly parsed in subsequent processing. The timestamps of all data streams are standardized according to the global network time base, and a high-precision synchronization mechanism is used to ensure the time consistency of data from each device. Then, based on predefined window size and sliding step parameters, a unified window division is performed on the multi-device data streams. The windowing process employs a combination of fixed and sliding windows, ensuring data aggregation at a fixed time granularity while allowing continuous analysis of data within time boundary regions. For each time window, communication records within the corresponding time range are extracted from standardized data streams from different devices, forming a set of time segments from multiple sources. Subsequently, the data streams within the time window are reorganized according to device topology, logically aggregating data from devices belonging to the same topological substructure for subsequent compression and correlation analysis. The windowing process is implemented using streaming processing, meaning data is assigned to the corresponding time window upon receipt, without waiting for the full dataset to arrive before unified processing.

[0024] In this embodiment, step S3 includes the following steps: Calculate the data packet arrival interval for different heterogeneous devices based on data streams from multiple time windows; Data stream analysis based on multiple time windows reveals device transmission patterns and periodic characteristics; Data stream difference analysis is performed on the data packet arrival interval, the device transmission mode, and periodic characteristics to obtain the data stream difference characteristics received by different devices; Entropy values ​​are calculated based on the data stream difference characteristics to obtain the data stream entropy values ​​of different devices. Adaptive encoding and multi-level compression are performed based on the entropy value of the data stream to construct a multi-device adaptive compression mechanism.

[0025] In this embodiment, for each device, all data packets within each time window are strictly sorted according to their arrival timestamps, and the difference in arrival times between adjacent data packets is calculated to obtain a series of interval time sequences. To eliminate the influence of clock drift and local abnormal fluctuations, global clock synchronization correction and moving average smoothing are performed on the timestamp data to ensure the consistency and stability of the calculated interval values. Subsequently, based on the window length and the statistical characteristics of device data traffic, multi-granularity aggregation analysis is performed on the interval sequences to extract statistics such as mean, variance, median, kurtosis, and skewness, forming transmission interval feature vectors for the device under different time windows. For sudden spikes in the interval sequences, correction is performed using a combination of median filtering and threshold suppression to avoid interference from outliers on the overall features. The interval sequences within each time window are segmented, and time-series clustering algorithms are used to identify communication behavior patterns at different stages. Next, Fast Fourier Transform (FFT) is used to perform frequency domain analysis on the interval sequences to extract the dominant frequency components and their amplitude distribution to determine whether there is significant periodicity in the device transmission behavior. For devices with a distinct dominant frequency, their dominant frequency position, amplitude ratio, and harmonic components are recorded to describe their periodic communication characteristics. For devices without a distinct dominant frequency, autocorrelation function and sliding window correlation detection are further used to determine whether their transmission mode is event-driven or aperiodic. In addition to frequency characteristics, the packet interval distribution pattern within each time window is statistically analyzed to distinguish between continuous, bursty, and mixed transmission modes. The entire analysis process employs a combination of batch processing and streaming analysis to ensure a complete characterization of the temporal characteristics of each device, even in large-scale, multi-device scenarios. The analysis results are ultimately converted into a standardized transmission mode description, including a dominant frequency feature vector, periodicity identifier, and transmission behavior type.

[0026] Using the interval characteristics, spectral characteristics, and mode labels of each device as input, a difference matrix between devices is constructed through multidimensional feature comparison and cluster analysis. The feature vectors of each device within different time windows are standardized and mapped to a unified feature space. Subsequently, various metrics such as Euclidean distance, Dynamic Time Warping (DTW) distance, and spectral similarity are used to compare the feature vectors of different devices pairwise and calculate the difference values. To more intuitively reveal the difference structure, a spectral clustering algorithm is used to partition the difference matrix, grouping devices with similar characteristics into the same group and retaining devices with significant feature differences as independent groups. In this process, not only the absolute numerical differences in interval and frequency characteristics are considered, but transmission mode labels are also used as classification constraints to ensure that the difference analysis fully reflects the essential differences between different types of communication behavior. The analysis results are output in the form of data stream difference features, including the difference index between each device, the classification results, and the identification information of key difference dimensions, providing a basis for subsequent entropy calculation and compression strategy selection. The data stream feature vectors of each device within each time window are discretized, transforming multi-dimensional information such as packet intervals, frequency peaks, and mode labels into discrete symbol sequences. Next, the Shannon entropy formula is used to calculate the probability distribution of the discrete symbols in each dimension, obtaining a single-dimensional entropy value for each feature dimension. For cases where different feature dimensions are correlated, the degree of interdependence is assessed through the calculation of joint entropy and mutual information, and weight adjustments are introduced in the final entropy value calculation to more accurately reflect the complexity of the overall data stream. The granularity of entropy value calculation for different devices is dynamically adjusted according to device type and topological location, ensuring comparability of entropy measurements between high-frequency communication devices and low-frequency event-type devices. Finally, each device obtains a comprehensive entropy value within each time window, representing the dynamic change and compressibility potential of its data stream. These entropy values ​​provide a basis for selecting adaptive coding strategies, enabling subsequent compression mechanisms to adopt differentiated processing strategies for different types of data streams. The entropy values ​​are hierarchically divided into high-entropy, medium-entropy, and low-entropy categories by setting multiple threshold intervals. For low-entropy data streams, static dictionary encoding and fixed-length compression are used to achieve high compression rates with minimal computational overhead. For medium-entropy data streams, adaptive Huffman coding or arithmetic coding is employed, dynamically generating the encoding table based on changes in data distribution to improve compression efficiency. For high-entropy data streams, a multi-level compression strategy is introduced: differential coding is first used to reduce the magnitude of data variation, followed by deep compression using a sliding window algorithm and context modeling to minimize data redundancy. The entire compression process adopts a hierarchical parallel processing structure, with different types of data streams processed simultaneously in independent compression channels and uniformly encapsulated and output after compression.Through an entropy-based adaptive coding mechanism, compression strategies can be flexibly adjusted according to the communication characteristics of different devices, enabling efficient compression of data from multi-protocol heterogeneous devices while ensuring real-time performance, thus providing a foundation for data optimization capabilities for subsequent cross-device interconnection and intelligent processing.

[0027] In this embodiment, the adaptive encoding and multi-level compression are specifically as follows: Define the entropy evaluation threshold; The entropy values ​​of the data streams are compared based on the entropy evaluation threshold to classify them into low-entropy data streams and high-entropy data streams. Dictionary encoding is used for low-entropy data streams; arithmetic encoding is used for high-entropy data streams. The multi-level compression includes protocol header compression, payload data compression, and cross-protocol associated compression.

[0028] In this embodiment, it is necessary to determine an entropy evaluation threshold applicable to multi-device, multi-protocol scenarios to classify and grade different types of data streams. The threshold definition process is based on the analysis of the entropy distribution characteristics of large-scale multi-source heterogeneous data. First, comprehensive entropy samples of each device within multiple time windows are collected and statistically modeled. The kernel density estimation method is used to obtain the overall entropy probability distribution curve. By observing the shape of the distribution curve, the concentrated distribution segments of data streams in low-entropy and high-entropy intervals are identified, and the initial boundary value is determined at the inflection point. Subsequently, the initial boundary value is dynamically adjusted according to the network scale, data stream type, and compression strategy objectives. The adjustment method introduces adaptive parameters and combines indicators such as the number of data streams, average packet length, and protocol layer number to weight and correct the threshold, thereby making the threshold adaptable to different network environments. To avoid classification bias caused by a single fixed threshold, an upper and lower dual threshold mechanism is also introduced, setting a low-entropy judgment threshold and a high-entropy judgment threshold to ensure that data streams within the boundary interval can flexibly select compression strategies according to the actual situation. The entropy value corresponding to each data stream is compared with the low and high thresholds one by one. When the entropy value is below the low threshold, the data stream is classified as a low-entropy data stream, indicating that its data characteristics are highly predictable and repetitive, suitable for low-overhead, high-efficiency encoding methods such as dictionary compression. When the entropy value is above the high threshold, it is classified as a high-entropy data stream, representing complex data changes, requiring more flexible arithmetic coding and other advanced compression algorithms for processing. For boundary data streams with entropy values ​​between the two thresholds, dynamic decisions are made based on their packet interval distribution, transmission mode type, and protocol characteristics, combined with priority strategies, to select a more suitable compression path. The classification process adopts a combination of batch processing and real-time processing. For high-frequency updated data streams, online classification is achieved through a streaming entropy value comparison mechanism; for stable data streams, classification is completed uniformly at the end of the window. By analyzing the repetition patterns of the low-entropy data stream content, a static or semi-dynamic compression dictionary is constructed. The dictionary contains frequently occurring field sequences, common protocol header patterns, and recurring transmission payload templates. During the encoding process, matching fragments in the original data are replaced with corresponding dictionary indices to achieve fast and high-ratio compression. Because low-entropy data streams have a relatively fixed structure, this encoding method can achieve a stable compression ratio while maintaining extremely low computational overhead. For high-entropy data streams, an arithmetic encoding method is used. First, the symbol distribution in the data stream is statistically modeled, and the probability interval of each symbol's occurrence is calculated. Then, the entire data sequence is mapped to a floating-point representation within the interval, achieving lossless compression. To cope with the dynamic changes in symbol distribution in high-entropy data, an adaptive probability update mechanism is introduced to correct symbol probabilities in real time during the encoding process, improving encoding efficiency and compression ratio.The dictionary encoding module and the arithmetic encoding module run in parallel. Low-entropy and high-entropy data streams enter their respective encoding channels simultaneously, and are finally encapsulated in a unified manner at the output stage to form a compressed multi-source data stream set.

[0029] By analyzing the redundant parts of various protocol header fields, fixed and predictable fields are reconstructed and differentially stored, retaining only the variable parts, significantly reducing the header space occupied. Secondly, payload data compression is performed. Appropriate compression algorithms are selected to process the payload independently based on the data stream type. For structured low-entropy payload data, block-level dictionary compression and duplicate pattern elimination are used; for high-entropy payloads, arithmetic coding or context modeling compression is continued. Finally, cross-protocol correlation compression is implemented. Based on the previously constructed heterogeneous device topology correlation structure and digital fingerprint map, the data stream correlation between different protocols is analyzed, identifying cross-protocol duplicate information, mapping fields, and time synchronization patterns. These cross-protocol redundant parts are stored through a correlation index table, achieving information sharing and compression between multiple protocols. The entire multi-level compression process adopts a staged pipeline structure, with each compression layer connected through standardized data interfaces, ensuring processing independence while facilitating flexible combination of different compression strategies. Through this multi-level compression mechanism, fine-grained compression can be implemented for data characteristics of different protocols and levels, significantly improving overall compression efficiency and data transmission performance, providing efficient data support for the intelligent interconnection of multi-protocol heterogeneous devices.

[0030] In this embodiment, step S4 includes the following steps: Identify the compression processing capability and storage capacity of each heterogeneous device; calculate the compression load of each device based on the compression processing capability and storage capacity, and generate the compression load characteristics of each device. Based on the compression load characteristics, the compression performance is evaluated to generate a device compression performance evaluation value. Identify the data stream to be transmitted from the current heterogeneous device; perform intelligent data segmentation processing on the data stream to be transmitted to obtain multiple data segments to be transmitted; The compression task is decomposed based on the multiple data segments to be transmitted, and multiple compression tasks are generated. Based on the equipment compression performance evaluation value, multiple compression tasks are dynamically allocated to obtain an intelligent task allocation strategy.

[0031] In this embodiment, a combination of active probing and passive monitoring is used. During the active probing phase, standardized performance probe request messages are sent to each device. Upon receiving these messages, the devices return response packets containing information such as their processor type, available computing resources (e.g., CPU clock speed, number of available cores, instruction set support), memory capacity, and current free storage space. During the passive monitoring phase, device operating status data is collected during normal communication, such as compression module runtime latency, buffer occupancy rate, and the execution time and completion rate of historical compression tasks. These parameters are continuously collected and updated by the monitoring agent of the edge nodes. The data obtained actively and passively are fused, and a normalization and feature weighting mechanism is used to standardize various capability indicators, forming a description vector of each device's processing capacity and storage capacity. The compression processing capability of the device is quantified using a weighted model, weighting factors such as computing power, instruction set support, and current CPU utilization to obtain a processing capacity index. Secondly, the device's storage capacity is dynamically calculated, comparing the remaining available storage space with the estimated cache usage of compression tasks to obtain a storage capacity utilization index. Then, the processing capacity index is combined with the storage capacity utilization rate, and a multi-index fusion algorithm is used to calculate the current compression load value. To reflect the load changes of the device in different time slices, a time-weighted sliding window is introduced into the compression load calculation, allowing the load characteristics to dynamically track the device status. Finally, a compression load feature vector is generated for each device, including key parameters such as processing load, storage utilization, task queuing latency, and compression response time. These features provide quantifiable inputs for subsequent compression performance evaluation.

[0032] Each dimension of the load feature vector is standardized to ensure comparability between parameters of different dimensions. Then, a performance evaluation model is established based on four core dimensions: compression efficiency, task response speed, storage availability, and parallel capability. Each dimension has an independent weight in the evaluation model, and the weights are dynamically adjusted based on device type, protocol type, and task characteristics. The load characteristics are matched with the evaluation model, and a weighted scoring and linear normalization method is used to output the final compression performance evaluation value. This evaluation value can be regarded as a comprehensive indicator of the device's ability to undertake compression tasks at the current moment. To avoid the impact of single fluctuations on evaluation stability, a historical moving average is introduced during the calculation process to smooth out short-term anomalies, making the evaluation value more reflective of the device's long-term compression performance. Through topology association and protocol adaptation mechanisms, the raw data stream information that all devices need to transmit at the current moment is collected, including data length, protocol type, time priority, transmission direction, and corresponding receiving node. Subsequently, a fragmentation strategy is selected based on the scale and characteristics of the data stream. The fragmentation process not only considers fixed-length partitions but also performs content-aware partitioning based on data type and compression method. For low-entropy data streams with high structure and obvious repetitive patterns, a larger fragment length is preferred to reduce fragment management overhead. For high-entropy and highly volatile data streams, a smaller fragment length is used to enable greater flexibility and parallelism in subsequent arithmetic coding processes. During fragmentation, the start and end positions, the device to which each fragment belongs, its priority, and the logical relationships between fragments are recorded simultaneously, forming a data fragment metadata structure. This intelligent data fragmentation method can divide the original heterogeneous data stream into multiple data segments suitable for parallel compression processing.

[0033] During task decomposition, each data segment is labeled with task attributes based on multi-dimensional information such as data segment size, protocol type, compression method, priority, and time requirements. For data segments requiring multi-level compression, they are further subdivided into header compression tasks, payload compression tasks, and cross-protocol associated compression tasks, with dependencies recorded in the task description. A task graph model is used to structurally represent all decomposed tasks, encoding the task order, dependencies, and parallel feasibility into a directed acyclic graph (DAG). This task graph serves as the core input for subsequent scheduling and allocation, enabling flexible allocation of compression tasks across different devices and supporting highly parallel compression processing. The granularity of task decomposition is dynamically adjusted based on device performance and network load, ensuring efficient task distribution while avoiding increased scheduling overhead due to excessive segmentation. All task and device evaluation values ​​are constructed into a two-sided matching model, where task requirements and device performance characteristics serve as nodes on both sides. A multi-objective optimization method is employed, comprehensively considering factors such as task completion time, compression efficiency, device load balancing, and data transmission latency to globally optimize the matching. First, by prioritizing tasks based on evaluation values, high-performance devices are preferentially allocated to high-complexity, high-entropy data compression tasks; low-performance devices are assigned to low-entropy, fixed-pattern tasks, thereby maximizing overall compression throughput. Second, a dynamic feedback mechanism is introduced during the allocation process to monitor task execution progress and device status changes in real time. If uneven load distribution or performance fluctuations occur, tasks are immediately reassigned and migrated. This dynamic scheduling mechanism ensures a continuous and efficient distribution of compression tasks across multiple devices, preventing overload or idle resources on some devices. Ultimately, this results in an adaptive, load-balanced intelligent task allocation strategy, enabling efficient collaborative processing of compression tasks in multi-protocol heterogeneous device environments.

[0034] In this embodiment, step S5 includes the following steps: Detect multi-dimensional network quality data of heterogeneous devices, optimize multi-path transmission load balancing, and construct multi-path transmission optimization strategies. Based on the intelligent task allocation strategy, real-time compression demand analysis is performed to obtain the compression demand value of the entire device. Based on the multi-path transmission optimization strategy and the compression requirements of all devices, the compression ratio and quality balance point are dynamically adjusted to obtain the intelligent compression adjustment strategy. Real-time intelligent data compression is driven by a multi-device adaptive compression mechanism and intelligent compression adjustment strategy.

[0035] In this embodiment, multi-dimensional network quality data is detected and collected for all devices involved in compression and data transmission. The detection includes metrics such as link bandwidth, average round-trip time (RTT), jitter rate, packet loss rate, number of available paths, path switching stability, and link reachability between the device and the cloud. The data collection method combines active probing and passive monitoring: in the active probing phase, probe packets are periodically sent to each device, and basic network metrics are calculated by analyzing response time and the integrity of returned data packets; in the passive monitoring phase, actual traffic statistics generated by the devices during normal transmission are used to capture link fluctuation characteristics in real time. The collected multi-dimensional network quality data, after time-weighted smoothing and outlier removal, is input into the multi-path transmission optimization module. This module uses a cost function-based load balancing algorithm to comprehensively quantify the bandwidth utilization, latency jitter, and reliability parameters of each path, forming a path performance score, and calculates the optimal task and data flow path allocation strategy through a multi-objective optimization method. Information such as the current compression task queue length, data fragment size, compression algorithm type, task priority, processing rate, and allocated transmission path performance metrics for each device is collected, and these data are statistically analyzed in real time based on a time window mechanism. By constructing a compression demand calculation model, the workload of each device (such as the amount of data to be compressed per unit time), the complexity of compression algorithms (such as the computational overhead of dictionary encoding and arithmetic encoding), task latency constraints, and device performance evaluation values ​​are combined to calculate the compression demand value of each device at the current moment. To more accurately reflect the overall situation, when calculating the compression demand value of all devices, the demand value of each device is weighted according to its position in the overall transmission topology, the importance of the task it undertakes, and the quality of the transmission path, forming a weighted total compression demand value.

[0036] The performance metrics of each path (such as bandwidth availability, packet loss rate, and latency stability) are matched and analyzed with the compression requirements of each device to identify areas with scarce and abundant network resources. For high-bandwidth, low-latency paths and devices with low compression requirements, the compression ratio is appropriately reduced to decrease compression overhead and retain more original data quality; while for low-bandwidth, high-latency paths or devices with high compression requirements, the compression ratio is appropriately increased to reduce transmission pressure. In the specific implementation, an adaptive ratio adjustment algorithm is adopted to find a balance between compression ratio and quality loss through dynamic iteration of the compression ratio and network transmission cost function. At the same time, according to task type and priority, higher compression ratios can be implemented for some non-critical data segments, while higher quality is retained for critical data segments. Finally, a set of intelligent compression adjustment strategies is generated, which clarifies the compression ratio and quality level configurations for each device, path, and task, and achieves coordinated optimization of compression resources and network resources. The adjustment strategy is distributed to each device and edge node participating in compression, updating their compression parameter configurations, including compression algorithm selection, compression ratio threshold, quality preservation strategy, and task scheduling priority. Subsequently, the compression execution engine adaptively processes the data streams entering the compression queue in real time based on task allocation results and network path optimization strategies. For low-entropy data streams, fast algorithms such as dictionary compression and header compression are prioritized; for high-entropy data streams, arithmetic coding or cross-protocol associative compression is selected according to the strategy. Simultaneously, during the compression process, the processing latency, compression throughput, and network transmission status of each device are continuously monitored. If network congestion, excessive device load, or changes in path performance are detected, dynamic readjustment of compression parameters is immediately triggered, ensuring that the compression mechanism remains synchronized with the real-time environment. The entire process employs a combination of distributed scheduling and centralized policy control, guaranteeing both local adaptive response speed and maintaining the consistency of the global compression strategy. Ultimately, high-efficiency, low-latency real-time intelligent data compression is achieved in a multi-protocol heterogeneous device network, providing stable compression performance for data transmission in complex environments.

[0037] In this embodiment, the specific steps for detecting multi-dimensional network quality data from heterogeneous devices, performing multi-path transmission load balancing optimization, and constructing a multi-path transmission optimization strategy are as follows: Detecting multidimensional network quality data from heterogeneous devices; Based on the multidimensional network quality data, bandwidth utilization, packet loss rate, round-trip time, and jitter are calculated, and network quality stability is assessed to obtain a network quality assessment report. Based on the network quality assessment report, network congestion trend analysis was conducted to obtain network congestion trend characteristics; Based on network congestion trend characteristics, transmission parameters are adjusted to obtain dynamic transmission adjustment parameters; The dynamic transmission adjustment parameters include retransmission timeout, transmission window size, and FEC redundancy. Multi-path transmission load balancing optimization is performed based on dynamic transmission adjustment parameters, and a multi-path transmission optimization strategy is constructed.

[0038] In this embodiment, in a multi-protocol heterogeneous device interconnection scenario, the network environments of different devices have differentiated characteristics, and their link quality is affected by various factors such as communication protocol type, access method, physical distance, network topology, and link load. Multi-dimensional network quality data detection is performed on all heterogeneous devices participating in data compression and transmission. The detection content includes multiple dimensions such as the actual available bandwidth of the link, round-trip time (RTT) of data packets, packet loss rate, jitter rate, path reachability, link stability, and the number of available paths. The detection method adopts a mechanism combining active probing and passive monitoring: In the active probing phase, probe packets are sent to each device at fixed time intervals. The RTT is calculated by comparing the timestamps of the returned packets, and packet loss is statistically analyzed based on the reception rate. Simultaneously, the available bandwidth is estimated by measuring the throughput changes of probe packets at different time periods. In the passive monitoring phase, during normal data flow transmission, the real-time status of the actual transmission link is continuously monitored, and indicators such as jitter characteristics, instantaneous bandwidth fluctuations, and abnormal retransmissions are extracted. All collected multi-dimensional network quality data undergoes time synchronization and outlier filtering processing, and is uniformly formatted before being input into the subsequent network quality evaluation module. Statistical analysis was performed on the collected bandwidth data to calculate the actual bandwidth utilization of each path and device within different time windows. During the calculation, the utilization rate was obtained by comparing the amount of data effectively transmitted per unit time with the theoretical maximum bandwidth of the link. For packet loss rate, the percentage of packet loss was calculated by comparing the number of probe packets with the number of actual received packets, and corrections were made for pseudo-packet loss caused by the retransmission mechanism. Round-trip time (RTT) was obtained by precisely matching the sending and receiving timestamps of each probe packet, and extreme outliers were further removed using a sliding window technique to ensure the stability of the delay assessment. Jitter was measured by calculating the variance or mean squared error of the RTT difference between adjacent data packets to measure the degree of delay fluctuation in the link. Next, time-series stability analysis was performed on these indicators, using statistical methods such as weighted mean, standard deviation, and coefficient of variation to quantitatively assess the long-term stability of the link. Finally, a network quality assessment report was generated, including a comprehensive score for each link, stability level, bottleneck indicator identification, and potential abnormal link markers, providing a reliable decision-making basis for subsequent congestion analysis and transmission parameter adjustment.

[0039] Key congestion-related indicators are extracted, such as the rate of increase in bandwidth utilization, the trend of packet loss rate, the continuous increase in round-trip latency, and the amplitude of jitter fluctuations. A time-series feature model is constructed, and sliding window statistics and trend fitting methods are used to model the trajectory of these indicators. Specifically, when bandwidth utilization continuously approaches the link's upper limit, packet loss rate shows a phased increase, and latency and jitter increase simultaneously, a congestion trend is identified. Simultaneously, a horizontal comparison mechanism between paths is introduced to identify whether congestion is localized (a problem on a specific path) or global (an increase in overall network load) by comparing the quality changes of multiple links within the same time period. For links with periodic fluctuation characteristics, periodic analysis algorithms (such as Fast Fourier Transform, FFT) are used to identify load peak periods, allowing for early adjustment of transmission strategies. After obtaining the network congestion trend characteristics, transmission parameters are dynamically adjusted according to different congestion levels and trend directions to adapt to changes in the network environment and improve transmission efficiency. The adjustment process mainly focuses on three core parameters: Retransmission Timeout (RTO), transmission window size, and Forward Error Correction (FEC) redundancy. For RTO, the timeout setting is adaptively adjusted based on the current average RTT and its fluctuation range to avoid premature retransmission during minor fluctuations and to prevent prolonged response time during congestion. For the transmission window size, the window is dynamically adjusted based on bandwidth utilization and packet loss rate, using a strategy similar to additive increase and multiplicative decrease (AIMD) to gradually expand the window when the link is smooth and quickly shrink it when congestion occurs, thereby reducing the impact of sudden congestion. For FEC redundancy, the number of redundant packets is determined by the short-term trend of packet loss rate changes to ensure improved link reliability without consuming excessive bandwidth. All parameter adjustments are performed within preset safety ranges to prevent excessive adjustments from causing adverse fluctuations.

[0040] For each available path, updated transmission parameters are applied, and the effective throughput and transmission cost of the path are recalculated to obtain a real-time path performance score. Then, a multi-objective optimization model is used to comprehensively determine the load distribution ratio of each path at the current moment, taking into account the path's bandwidth utilization, transmission latency, jitter, and redundancy overhead caused by FEC. For high-quality, low-latency paths, more high-priority or time-sensitive data streams are allocated; for lower-quality paths, data segments with strong fault tolerance or low priority are allocated. Simultaneously, the operating status of each path is continuously monitored, and when a significant fluctuation in path performance is detected, a load reallocation mechanism is automatically triggered to achieve dynamic balancing. A multi-path transmission optimization strategy is constructed, enabling intelligent distribution of data streams across multiple paths based on real-time network status and transmission parameters, effectively improving overall transmission stability and efficiency.

[0041] In this embodiment, a data intelligent compression system based on multi-protocol heterogeneous device interconnection is provided for executing the data intelligent compression method based on multi-protocol heterogeneous device interconnection as described above, including: The protocol parsing module is used to perform real-time parsing and protocol conversion and adaptation when a new heterogeneous device is detected requesting to join the network, so as to obtain the heterogeneous device adaptation protocol. The data stream segmentation module is used to receive data streams from multiple devices according to the heterogeneous device adaptation protocol and generate data streams with multiple time windows. The multi-level compression module is used to adaptively encode and compress data streams from multiple time windows, and to build a multi-device adaptive compression mechanism. The intelligent task allocation module is used to identify the compression load characteristics of each heterogeneous device, perform dynamic task allocation, and thus obtain an intelligent task allocation strategy. The collaborative control module is used to drive real-time intelligent data compression operations based on intelligent task allocation strategies and multi-device adaptive compression mechanisms.

[0042] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.

[0043] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein are implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.

Claims

1. A data intelligent compression method based on the interconnection of multi-protocol heterogeneous devices, characterized in that, Includes the following steps: Step S1: When a new heterogeneous device is detected requesting to join the network, real-time parsing and protocol conversion and adaptation are performed to obtain the heterogeneous device adaptation protocol; Step S2: Receive data streams from multiple devices according to the heterogeneous device adaptation protocol, and generate data streams for multiple time windows; Step S3: Perform adaptive encoding and multi-level compression on data streams from multiple time windows to build a multi-device adaptive compression mechanism; Step S4: Identify the compression load characteristics of each heterogeneous device, perform dynamic task allocation, and thus obtain an intelligent task allocation strategy; Step S5: Drive real-time intelligent data compression operations based on intelligent task allocation strategies and multi-device adaptive compression mechanisms.

2. The intelligent data compression method based on the interconnection of multi-protocol heterogeneous devices according to claim 1, characterized in that, The specific steps of step S1 are as follows: Real-time monitoring of network space status; when a new heterogeneous device requests to join the network, preload containerized parsing units from the cloud protocol library. The containerized parsing unit performs real-time parsing of the communication protocol of the heterogeneous device to extract the communication protocol information of the heterogeneous device. The communication protocol information includes the protocol header structure, data payload mode and timing features. The communication protocol information is analyzed for device type to obtain the device type identifier; Based on the communication protocol information, analyze the protocol stack hierarchy and data transmission frequency mode, and combine the device type identifier to perform device digital fingerprint evolution, and construct a heterogeneous device digital fingerprint map. Based on the digital fingerprint map of heterogeneous devices, protocol conversion and adaptation are performed in the network space to obtain the heterogeneous device adaptation protocol.

3. The intelligent data compression method based on the interconnection of multi-protocol heterogeneous devices according to claim 1, characterized in that, The specific steps of step S2 are as follows: Identify heterogeneous connected devices in cyberspace, perform device interaction analysis, and extract physical connection relationships, logical dependencies, and data flow paths between devices; Based on the physical connection relationship, logical dependency relationship and data flow path, the device topology association is parsed to obtain the heterogeneous device topology association structure; Based on the heterogeneous device topology and heterogeneous device adaptation protocol, multi-device data stream reception is performed, and data streams from different heterogeneous devices are received in advance. The data stream is divided into time-series windows to generate data streams with multiple time windows.

4. The intelligent data compression method based on the interconnection of multi-protocol heterogeneous devices according to claim 1, characterized in that, Step S3 is as follows: Calculate the data packet arrival interval for different heterogeneous devices based on data streams from multiple time windows; Data stream analysis based on multiple time windows reveals device transmission patterns and periodic characteristics; Data stream difference analysis is performed on the data packet arrival interval, the device transmission mode, and periodic characteristics to obtain the data stream difference characteristics received by different devices; Entropy values ​​are calculated based on the data stream difference characteristics to obtain the data stream entropy values ​​of different devices. Adaptive encoding and multi-level compression are performed based on the entropy value of the data stream to construct a multi-device adaptive compression mechanism.

5. The intelligent data compression method based on multi-protocol heterogeneous device interconnection according to claim 4, characterized in that, The adaptive coding and multi-level compression are specifically as follows: Define the entropy evaluation threshold; The entropy values ​​of the data streams are compared based on the entropy evaluation threshold to classify them into low-entropy data streams and high-entropy data streams. Dictionary encoding of low-entropy data streams; Arithmetic encoding of high-entropy data streams; The multi-level compression includes protocol header compression, payload data compression, and cross-protocol associated compression.

6. The intelligent data compression method based on the interconnection of multi-protocol heterogeneous devices according to claim 1, characterized in that, The specific steps of step S4 are as follows: Identify the compression processing capability and storage capacity of each heterogeneous device; calculate the compression load of each device based on the compression processing capability and storage capacity, and generate the compression load characteristics of each device. Based on the compression load characteristics, the compression performance is evaluated to generate a device compression performance evaluation value. Identify the data streams to be transmitted from the current heterogeneous devices; The data stream to be transmitted is subjected to intelligent data fragmentation processing to obtain multiple data segments to be transmitted; The compression task is decomposed based on the multiple data segments to be transmitted, and multiple compression tasks are generated. Based on the equipment compression performance evaluation value, multiple compression tasks are dynamically allocated to obtain an intelligent task allocation strategy.

7. The intelligent data compression method based on the interconnection of multi-protocol heterogeneous devices according to claim 1, characterized in that, The specific steps of step S5 are as follows: Detect multi-dimensional network quality data of heterogeneous devices, optimize multi-path transmission load balancing, and construct multi-path transmission optimization strategies. Based on the intelligent task allocation strategy, real-time compression demand analysis is performed to obtain the compression demand value of the entire device. Based on the multi-path transmission optimization strategy and the compression requirements of all devices, the compression ratio and quality balance point are dynamically adjusted to obtain the intelligent compression adjustment strategy. Real-time intelligent data compression is driven by a multi-device adaptive compression mechanism and intelligent compression adjustment strategy.

8. The intelligent data compression method based on multi-protocol heterogeneous device interconnection according to claim 7, characterized in that, The specific steps for constructing a multi-path transmission optimization strategy by analyzing the multi-dimensional network quality data of heterogeneous devices are as follows: Detecting multidimensional network quality data from heterogeneous devices; Based on the multidimensional network quality data, bandwidth utilization, packet loss rate, round-trip time, and jitter are calculated, and network quality stability is assessed to obtain a network quality assessment report. Based on the network quality assessment report, network congestion trend analysis was conducted to obtain network congestion trend characteristics; Based on network congestion trend characteristics, transmission parameters are adjusted to obtain dynamic transmission adjustment parameters; The dynamic transmission adjustment parameters include retransmission timeout, transmission window size, and FEC redundancy. Multi-path transmission load balancing optimization is performed based on dynamic transmission adjustment parameters, and a multi-path transmission optimization strategy is constructed.

9. A data intelligent compression system based on the interconnection of multi-protocol heterogeneous devices, characterized in that, The method for performing intelligent data compression based on multi-protocol heterogeneous device interconnection as described in claim 1 includes: The protocol parsing module is used to perform real-time parsing and protocol conversion and adaptation when a new heterogeneous device is detected requesting to join the network, so as to obtain the heterogeneous device adaptation protocol. The data stream segmentation module is used to receive data streams from multiple devices according to the heterogeneous device adaptation protocol and generate data streams with multiple time windows. The multi-level compression module is used to adaptively encode and compress data streams from multiple time windows, and to build a multi-device adaptive compression mechanism. The intelligent task allocation module is used to identify the compression load characteristics of each heterogeneous device, perform dynamic task allocation, and thus obtain an intelligent task allocation strategy. The collaborative control module is used to drive real-time intelligent data compression operations based on intelligent task allocation strategies and multi-device adaptive compression mechanisms.

Images