Self-adaptive compression and lossless fast transmission method and system for high-frequency test data

Abstract

The invention discloses a self-adaptive compression and lossless fast transmission method and system for high-frequency test data. The method comprises the following steps: extracting multi-dimensional dynamic feature vectors from high-frequency test data in real time and marking feature points; based on the features, the network bandwidth and the target compression ratio, dynamically selecting a compression algorithm combination and parameters through a pre-training decision model; performing data framing and typed preprocessing according to the algorithm combination; compression is executed through a multi-stage assembly line, and a correction link based on statistical characteristics is inserted between stages; constructing a three-dimensional transmission state matrix fusing priorities, links and quality scores, dynamically allocating links and adaptively adding forward error correction codes; and after error correction and decompression of a receiving end, realizing lossless verification through multi-dimensional comparison of feature point sequences of the reconstructed data and the original data. According to the method, the whole-course self-adaption of the compression strategy and the transmission scheduling is realized, and the lossless and reliable recovery of the data is ensured while the compression and transmission efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of high-frequency test data processing, and more specifically, to an adaptive compression and lossless fast transmission method and system for high-frequency test data. Background Technology

[0002] High-frequency test data refers to data streams of continuously varying digital sequences of physical quantities acquired by sensors or acquisition devices in industrial monitoring, scientific research, equipment testing, or power systems. These data streams typically exhibit high sampling rates (tens of thousands to millions of times per second) and change continuously over time. They are characterized by high data rates, high information density, and stringent real-time requirements. The acquisition and analysis of this data are crucial for equipment condition monitoring, product quality assessment, and scientific research. However, the massive amounts of raw data pose significant challenges to storage space, transmission bandwidth, and post-processing capabilities. Therefore, efficient data compression and reliable transmission technologies are key to this field.

[0003] In data compression, existing technologies typically employ fixed, general-purpose compression algorithms (such as ZIP and RAR) or dedicated coding for specific signal types. General-purpose compression algorithms have limited efficiency when processing complex, high-frequency test data, making it difficult to balance compression ratio and speed. Dedicated coding, on the other hand, lacks universality, and its compression performance significantly degrades when data characteristics (such as stationarity, periodicity, and transient properties) change. More advanced methods incorporate signal processing techniques for preprocessing, but preprocessing rules are often static or semi-static, unable to be adjusted online according to the real-time dynamic characteristics of the data stream. This results in poor compression performance for non-stationary, non-periodic, and transient data changes, or sacrifices overall compression ratio to preserve detail.

[0004] In terms of data transmission, existing solutions mostly focus on optimizing a single network link, such as adjusting the packet transmission rate or adding fixed redundancy error correction codes. However, in complex industrial network environments, multiple available links (such as wired, wireless, and 5G in different frequency bands) often exist, with their latency, packet loss rate, and bandwidth dynamically changing. Existing methods lack the ability to coordinate and schedule resources across multiple links, and cannot perform intelligent routing and differentiated protection based on the importance of data packets and the real-time network status. This often results in critical data being lost or experiencing excessive delays when transmitted on inferior links, while non-critical data occupies high-quality link resources, making it impossible to balance overall transmission efficiency and reliability.

[0005] Furthermore, in ensuring data integrity, existing technologies largely rely on the reliability guarantees of transport layer protocols (such as TCP) or application layer verification. However, TCP's retransmission mechanism introduces uncontrollable delays under high-speed data streams, making it unsuitable for strictly real-time scenarios; while simple checksums can only detect errors, but cannot locate and correct them, let alone verify whether the decompressed and reconstructed data is completely consistent with the original data in terms of numerical precision, thus failing to achieve end-to-end lossless verification from the physical layer to the application layer.

[0006] Based on the above analysis, existing technologies for high-frequency test data compression and transmission mainly suffer from the following three pressing technical problems: First, the compression methods lack adaptability, failing to dynamically select and adjust the optimal compression strategy according to the real-time dynamic characteristics of the data stream and network bandwidth constraints, resulting in low compression efficiency. Second, the transmission process lacks intelligent perception and collaborative scheduling of multi-link environments, making it difficult to simultaneously achieve high-speed (fast transmission) and highly reliable (lossless) data transmission under complex network conditions. Third, the verification mechanism is imperfect, lacking a closed-loop method that combines transmission error correction with content accuracy verification, failing to ensure true lossless recovery of data after the entire compression-transmission-decompression process. Therefore, an adaptive compression and lossless fast transmission method and system for high-frequency test data are urgently needed to address the aforementioned problems. Summary of the Invention

[0007] This application provides an adaptive compression and lossless fast transmission method and system for high-frequency test data, which at least addresses the problems existing in the prior art.

[0008] A first aspect of this application provides an adaptive compression and lossless fast transmission method for high-frequency test data, comprising the following steps: S1. Perform real-time time-frequency domain analysis on the input high-frequency test data, extract multi-dimensional dynamic feature vectors, and mark the feature point sampling values; S2. Based on the extracted multidimensional dynamic feature vector, and combined with the current available network bandwidth and the target compression ratio, dynamically select the compression combination algorithm; S3. Process the high-frequency test data according to the compression and combination algorithm; S4. Execute the compression and combination algorithm, and use the output of the previous algorithm as the input of the next algorithm. Insert an intermediate correction step between adjacent algorithms to adjust the input parameters of the next algorithm to obtain the compressed data packet. S5. Construct a transmission state matrix based on transmission priority, network latency, and packet loss rate. Dynamically allocate compressed data packets to different network links based on the transmission state matrix and add redundant forward error correction codes to achieve lossless fast transmission of high-frequency test data. S6. After receiving compressed data packets on different network links, decompression is performed in reverse, and lossless verification of high-frequency test data is achieved by comparing the feature point sampling values ​​with the redundant forward error correction code.

[0009] This application solves the technical problems of traditional methods, such as the disconnect between compression and transmission, the inability to dynamically adjust based on data and network conditions, leading to low efficiency or the inability to guarantee lossless recovery, by first extracting data features and intelligently selecting compression strategies, and then optimizing transmission and verification in conjunction with network conditions. It achieves the goal of balancing high compression ratio, fast transmission speed, and data integrity in complex network environments.

[0010] In some embodiments of this application, in step S1, the real-time time-frequency domain analysis includes intercepting the data stream through a sliding window, performing short-time Fourier transform and discrete wavelet transform on the data in each window in parallel; calculating the concentration of signal energy on the first N principal components from the short-time Fourier transform results as a spectral feature, calculating Shannon entropy from the multi-scale coefficients of the discrete wavelet transform as a complexity feature, and calculating the average value of the absolute difference of the data in the window in the time domain as a data rate of change feature, while identifying and recording the positions and values ​​of local extreme points exceeding a preset threshold in the window as feature point sampling values, where N is a positive integer.

[0011] This application performs short-time Fourier transform and discrete wavelet transform on the sliding window data in parallel, and comprehensively calculates multi-dimensional features such as spectral concentration, Shannon entropy, and rate of change, thereby comprehensively and accurately characterizing the time-frequency domain characteristics and transient details of high-frequency test signals. This solves the problem of insufficient descriptive ability of traditional single feature extraction methods, providing a richer and more reliable basis for subsequent adaptive compression decisions, thus improving the overall adaptability and efficiency of compression.

[0012] In some embodiments of this application, the dynamic selection of compression combination algorithm in step S2 specifically involves: inputting the multidimensional dynamic feature vector extracted in step S1, along with the current available network bandwidth and the target compression ratio, into a pre-trained classification and regression decision tree model; the classification and regression decision tree model first outputs a compression mode classification, which corresponds to a compression algorithm sequence framework, and then outputs a set of specific parameter configurations for each algorithm step in the compression algorithm sequence framework, thereby realizing the dynamic selection of compression combination algorithm based on the specific parameter configurations.

[0013] This application employs a pre-trained classification and regression decision tree model, using multi-dimensional features, network bandwidth, and the target compression ratio as inputs to achieve integrated dynamic selection from compression mode to specific parameters. This solves the problem of rigid, manually pre-configured compression strategies being unable to adapt to changing data patterns, realizing intelligent and automated generation of compression strategies, and significantly improving the compression process's adaptability to different data characteristics and overall compression performance.

[0014] In some embodiments of this application, step S2 further includes online fine-tuning of the decision tree model. During the compression process, the actual compression ratio and compression speed are collected in real time. When the deviation between the actual compression ratio and the target compression ratio continues to exceed the set range, or the compression speed is lower than expected, the decision threshold or output value of the corresponding node of the current decision tree model is corrected online.

[0015] This application introduces an online fine-tuning mechanism for the decision model based on real-time compression performance feedback, enabling the system to continuously learn and optimize during operation. This addresses the issue of static models potentially failing when faced with changes in local characteristics of the data stream or environmental fluctuations, enhancing the system's robustness and adaptability, and ensuring that compression performance remains stable near its optimal state over the long term.

[0016] In some embodiments of this application, step S3 processes the high-frequency test data according to the compression and combination algorithm, specifically including: dividing the high-frequency test data into overlapping frames and non-overlapping frames according to the data continuity requirements of the selected compression and combination algorithm; and labeling each frame of data with a type label based on spectral characteristics, complexity characteristics, and data change rate characteristics. The type labels include stationary frames, transient frames, and periodic frames.

[0017] This application provides structured data units and clear processing guidance by framing and labeling the raw data according to the requirements of the compression algorithm and data characteristics. This solves the problem of difficulty in implementing targeted optimization strategies when processing data streams as a whole, achieves preliminary data classification and management, and lays the foundation for implementing differentiated and efficient preprocessing.

[0018] In some embodiments of this application, step S3 further includes a differential preprocessing step corresponding to the frame type label, specifically including: for data labeled as stationary frames, a moving average filter is used for smoothing to reduce random noise; for data labeled as transient frames, an edge-preserving filter is used for noise reduction while maintaining the edges of abrupt changes, and the start and end points of abrupt changes are marked with high precision; for data labeled as periodic frames, a base period estimation and phase alignment operation based on the autocorrelation function is performed to improve the efficiency of the periodic compression algorithm.

[0019] This application employs specific preprocessing operations for different frame types, such as stationary, transient, and periodic frames, including moving average smoothing, edge-preserving noise reduction and labeling, and period estimation and phase alignment. This solves the problem that traditional single preprocessing methods may damage the characteristics of specific data types (such as blurring edges or destroying periodicity). It can remove noise interference while preserving or enhancing data structures that are beneficial to subsequent compression to the greatest extent, thereby significantly improving the compression potential of various types of data.

[0020] In some embodiments of this application, the intermediate correction step in step S4 specifically includes: performing statistical characteristic analysis on the output data block of the previous algorithm, calculating its data distribution histogram, mean, and variance; comparing the statistical characteristics with the ideal input data distribution range preset for the next algorithm in the currently selected compression combination algorithm; if there is a deviation, transforming the data block through a numerical mapping function to adjust its statistical characteristics to the preset ideal range, and then inputting the adjusted data block into the next algorithm.

[0021] This application introduces a correction mechanism based on statistical characteristic analysis between compression pipeline stages. Through numerical mapping, it adjusts the output data of the preceding stage to the ideal input range of the subsequent algorithm. This solves the problem that when multiple compression algorithms are cascaded, the output of the preceding stage may deviate from the efficient operating range of the subsequent algorithm, leading to decreased compression efficiency or even distortion. It ensures that each stage of the algorithm operates at its optimal state, improving the stability and output quality of the overall compression pipeline.

[0022] In some embodiments of this application, the transmission state matrix in step S5 is a three-dimensional matrix, whose three dimensions represent different transmission priority levels of data packets, different alternative network link identifiers, and a link quality score calculated by combining real-time network latency, packet loss rate, and estimated throughput. According to the transmission priority of the data packets, the corresponding two-dimensional sub-matrix is ​​selected in the matrix, and then a preset algorithm is used to select the link with the best link quality score and not currently under full load for each data packet in the two-dimensional sub-matrix for binding.

[0023] This application achieves refined management of data packet transmission by constructing a three-dimensional transmission state matrix that integrates transmission priority, link identification, and link quality score, and by performing optimal link matching based on a priority selection submatrix. This solves the problem in traditional transmission scheduling of the difficulty in coordinating differences in data importance and the variability of network link states, and can intelligently guide important data to high-quality links, thereby optimizing overall transmission efficiency and reliability.

[0024] In some embodiments of this application, the process of adding redundant forward error correction codes in step S5 is dynamically associated with link allocation. First, the system maintains a lookup table of link packet loss rate and required error correction redundancy. After allocating a specific link to a data packet, the system immediately queries the real-time packet loss rate of that link and dynamically determines the type and redundancy of the error correction code to be added based on the lookup table. For high-priority data packets allocated to links with high packet loss rates, a coding method with strong error correction capability but high overhead is adopted. For low-priority data packets allocated to high-quality links, lightweight verification or even no error correction code is added.

[0025] This application dynamically correlates the redundancy of forward error correction codes with the real-time packet loss rate of the specific link allocated to the data packet, establishing a differentiated error protection strategy. This solves the problem that fixed error correction schemes either provide insufficient protection leading to retransmissions or over-protect and waste bandwidth in variable network environments, achieving an adaptive balance between network resources and reliability requirements, maximizing effective data transmission efficiency while ensuring the reliability of critical data.

[0026] In some embodiments of this application, the receiving end first uses redundant forward error correction codes to detect and correct errors in the received data packets. Correctable errors are directly repaired, while errors exceeding the error correction capability are marked as suspicious packets. Based on the corrected data and compression strategy information, preliminary decompression is performed to obtain the reconstructed data stream to be verified. According to the feature point location index information associated with the sending end and the compressed data stream, each feature point is accurately located in the reconstructed data stream to be verified. The actual sampled values ​​at the corresponding positions in the reconstructed data stream are extracted to form a sequence of reconstructed feature point sampled values. The sequence of reconstructed feature point sampled values ​​is compared with the original feature point sampled value sequence synchronized by the sending end. The comparison dimensions include numerical absolute error, local waveform similarity, and statistical distribution consistency. Only when the comparison results of all dimensions meet their respective preset tolerance thresholds is the transmission and reconstruction process of the batch of data determined to be lossless. If any dimension comparison fails, a retransmission request for the suspicious data packet or the corresponding data segment is triggered.

[0027] This application provides a complete lossless verification process, from error correction and recovery, initial decompression, precise feature point localization and extraction to multi-dimensional comparison. This solves the problems that a single verification method (such as CRC) may not be able to detect all types of distortion, and that traditional methods are difficult to verify the consistency of data content features. Through a rigorous, multi-layered verification mechanism, it ensures that data is only deemed lossless when both content and features are strictly consistent, greatly improving the reliability and authority of data integrity verification.

[0028] A second aspect of this application provides a system for adaptive compression and lossless fast transmission of high-frequency test data, comprising: a dynamic feature extraction module for performing real-time time-frequency domain analysis on the input high-frequency test data, extracting multi-dimensional dynamic feature vectors, and marking feature point sampling values; a compression decision module for dynamically selecting a compression combination algorithm based on the extracted multi-dimensional dynamic feature vectors, combined with the currently available network bandwidth and the target compression ratio; a data preprocessing module for processing the high-frequency test data according to the compression combination algorithm; a multi-level compression execution module for executing the compression combination algorithm, using the output of the previous level algorithm as the input of the next level algorithm, and inserting an intermediate correction step between adjacent algorithms to adjust the input parameters of the next level algorithm to obtain compressed data packets; a transmission scheduling and protection module for constructing a transmission state matrix based on transmission priority, network latency, and packet loss rate, dynamically allocating compressed data packets to different network links based on the transmission state matrix, and adding redundant forward error correction codes; and a lossless verification and reconstruction module for performing decompression in reverse after receiving compressed data packets on different network links, and realizing lossless verification of high-frequency test data by comparing feature point sampling values ​​with redundant forward error correction codes.

[0029] A third aspect of the present application provides an electronic device having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the adaptive compression and lossless fast transmission method for high-frequency test data as described above.

[0030] In the above embodiments, the system claims protect the hardware architecture and software module entities that implement the above method. Through the synergistic effect of the various modules, efficient, reliable, and scalable physical support is provided for the implementation of the method. Attached Figure Description

[0031] Figure 1 A flowchart illustrating the adaptive compression and lossless fast transmission method for high-frequency test data provided in this application embodiment; Figure 2 A schematic diagram of an adaptive compression and lossless fast transmission system for high-frequency test data provided in an embodiment of this application; Figure 3 A schematic diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0032] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0033] To make the purpose, technical solution, and advantages of this application clearer, the following will be described in conjunction with the appendix. Figure 1-3 The following is an explanation using specific examples.

[0034] Please refer to Figure 1 , Figure 1 This application provides an adaptive compression and lossless fast transmission method for high-frequency test data, comprising the following steps: S1. Perform real-time time-frequency domain analysis on the input high-frequency test data, extract multi-dimensional dynamic feature vectors, and mark the feature point sampling values; S2. Based on the extracted multidimensional dynamic feature vector, and combined with the current available network bandwidth and the target compression ratio, dynamically select the compression combination algorithm; S3. Process the high-frequency test data according to the compression and combination algorithm; S4. Execute the compression and combination algorithm, and use the output of the previous algorithm as the input of the next algorithm. Insert an intermediate correction step between adjacent algorithms to adjust the input parameters of the next algorithm to obtain the compressed data packet. S5. Construct a transmission state matrix based on transmission priority, network latency, and packet loss rate. Dynamically allocate compressed data packets to different network links based on the transmission state matrix and add redundant forward error correction codes to achieve lossless fast transmission of high-frequency test data. S6. After receiving compressed data packets on different network links, decompression is performed in reverse, and lossless verification of high-frequency test data is achieved by comparing the feature point sampling values ​​with the redundant forward error correction code.

[0035] As can be understood, real-time time-frequency domain analysis refers to the simultaneous joint mathematical transformation of high-frequency test signals that change rapidly over time, performing both time and frequency dimensions. It goes beyond simply observing the amplitude changes over time; instead, it uses methods such as short-time Fourier transform or wavelet transform to decompose the signal into a two-dimensional time-frequency plane, clearly revealing how the frequency components of the signal evolve over time. This is crucial for capturing transient changes, identifying periodic patterns, and analyzing the local stability of signals, forming the foundation for understanding the essential characteristics of complex test data. A multidimensional dynamic feature vector is a set of values ​​quantified and extracted from the above time-frequency domain analysis results, comprehensively characterizing the instantaneous state and changing trends of the signal. It is usually not a single indicator but integrates features such as the drastic changes in the signal per unit time, the frequency bands where the main energy is concentrated, the irregularity or complexity of the waveform, and the duration of specific events such as peak duration. These feature values ​​together form a vector, serving as a digital fingerprint for intelligently judging and classifying the current data segment. Feature point sampling values ​​refer to the signal amplitude values ​​at discrete time points selected from the original data stream that have significant representativeness or special meaning. These points are typically locations where signals undergo abrupt changes, local peaks or troughs, or the starting points of specific patterns. Recording the locations and corresponding precise values ​​of these points aims to provide a crucial and tamper-proof benchmark for subsequent lossless data verification. A compression combinatorial algorithm refers to a processing flow that combines multiple basic compression coding methods in a certain logical order to achieve a specific compression goal. It is not a single algorithm but a strategic pipeline; for example, one algorithm might be used first to remove correlations between data, and then another algorithm might be used to efficiently encode the remaining information. Dynamic selection means that the system can select and assemble the most suitable processing chain from the algorithm library in real time based on data characteristics and external conditions. The intermediate correction stage is an adaptive adjustment step inserted between two stages of a multi-stage compression pipeline. Its core function is to monitor the statistical characteristics of the intermediate data output by the previous stage algorithm, such as its numerical distribution range or central tendency, and compare it with the ideal input characteristics expected by the next stage algorithm. Once a deviation is detected, the intermediate data is slightly corrected through specific mathematical transformations to better match the processing requirements of subsequent algorithms. This ensures that the entire compression pipeline can work efficiently and stably, avoiding a decrease in compression efficiency due to inter-stage mismatch. The transmission state matrix is ​​a multi-dimensional data structure used to assist transmission decisions. It abstractly integrates various aspects of state information; its different dimensions can represent the priority category of the data packets to be transmitted, the currently available physical or logical network channels, and the real-time quality assessment score of each channel. This matrix provides the transmission scheduler with a global perspective, enabling it to make optimal or near-optimal transmission path selections based on the importance of the data packets and the real-time status of the network links.Redundant forward error correction (FEC) is a technique for error control that adds extra checksum information to the data to be transmitted. Its core idea is that the sending end generates and adds redundant bits according to specific rules. The receiving end uses these redundant bits to not only detect whether errors occur during data transmission, but also to automatically correct errors within a certain error range without requesting retransmission from the sending end. This mechanism sacrifices some bandwidth efficiency but significantly improves transmission reliability, making it crucial for ensuring data arrives intact in unreliable channels. Reverse decompression is the strict reverse operation of the sending end's compression process. After receiving the compressed data packet and related compression strategy description information, the receiving end needs to strictly follow the reverse process of the algorithm used during compression to gradually reconstruct the data. For example, if the last step in compression is entropy encoding, then the first step in decompression is the corresponding entropy decoding. This process must be precise and error-free; any error in any step or parameter may lead to distortion of the final reconstructed data. Lossless verification is a comprehensive judgment step to confirm that no information loss has been introduced into the entire compression, transmission, and decompression process. It not only relies on error correction codes at the communication link layer to ensure bit correctness but also performs final verification at the application layer by comparing the consistency of the reconstructed data and the original data at key feature points. Only when the sampled values ​​of feature points extracted from the reconstructed data completely match the original values ​​pre-saved at the sending end within the allowable error range can it be concluded from a mathematical and engineering perspective that the data has achieved true lossless recovery.

[0036] This application solves the technical problems of traditional methods, such as the disconnect between compression and transmission, the inability to dynamically adjust based on data and network conditions, leading to low efficiency or the inability to guarantee lossless recovery, by first extracting data features and intelligently selecting compression strategies, and then optimizing transmission and verification in conjunction with network conditions. It achieves the goal of balancing high compression ratio, fast transmission speed, and data integrity in complex network environments.

[0037] In some embodiments of this application, in step S1, the real-time time-frequency domain analysis includes intercepting the data stream through a sliding window, performing short-time Fourier transform and discrete wavelet transform on the data in each window in parallel; calculating the concentration of signal energy on the first N principal components from the short-time Fourier transform results as a spectral feature, calculating Shannon entropy from the multi-scale coefficients of the discrete wavelet transform as a complexity feature, and calculating the average value of the absolute difference of the data in the window in the time domain as a data rate of change feature, while identifying and recording the positions and values ​​of local extreme points exceeding a preset threshold in the window as feature point sampling values, where N is a positive integer.

[0038] Understandably, a sliding window is a common segmentation technique for processing continuous data streams. It divides a long data sequence into a series of continuous or partially overlapping fixed-length or variable-length segments. Each such segment is called a window, and analysis is performed independently within each window, thus achieving localized data processing and allowing features to be tracked over time. The Short-Time Fourier Transform (SFT) is a generalization of the traditional Fourier Transform. It applies a sliding, finite-time window to the signal and then performs a Fourier transform on the signal within each window, resulting in a rough representation of the signal's frequency content over time—the spectrum. The Discrete Wavelet Transform (DWT) uses a series of scalable and shiftable wavelet basis functions for multi-resolution analysis of the signal. It provides good localization characteristics in both the time and frequency domains, making it particularly suitable for analyzing non-stationary signals and capturing transient features. Principal component concentration is a measure obtained by processing the spectrum using principal component analysis. It reflects the degree to which signal energy is concentrated on a few main frequency components; higher concentration usually indicates a simpler spectrum. In this context, Shannon entropy is a quantification of the disorder or irregularity of the distribution of wavelet coefficients across various scales; a higher entropy value indicates higher local complexity of the signal. The average of the absolute values ​​of data differences is calculated by summing the absolute values ​​of the differences between adjacent data points within a window and then averaging them; it intuitively measures the average drasticness of signal changes over that time period. Local extrema refer to those data points within a window whose amplitudes are greater than those of all their neighbors—i.e., peak points within a local range.

[0039] The principle behind this step in this embodiment lies in comprehensively and quantitatively characterizing the intrinsic properties of a short-time data window from different perspectives through parallel time-frequency transformation and multi-dimensional statistics, providing a comprehensive feature basis that integrates time domain, frequency domain, and information theory for subsequent intelligent decision-making. The sliding window mechanism ensures the local adaptability and real-time performance of the analysis. Parallel short-time Fourier transform and discrete wavelet transform on the window data constitute the dual foundation for feature extraction. The short-time Fourier transform provides a global overview of the evolution of the signal's frequency content over time. By calculating the concentration of its energy on the first N principal components, concise core features of the signal's spectral structure can be extracted, such as whether it is dominated by a single frequency or has a wide bandwidth distribution. The discrete wavelet transform provides a multi-scale detailed view of the signal. By calculating the Shannon entropy of its coefficients at each scale, the complexity and irregularity of the signal at different resolutions can be measured, which is crucial for identifying the transients, noise levels, or fine structure of the signal. At the same time, in the pure time domain, by calculating the average of the absolute values ​​of the differences, a robust estimate of the signal's rate of change within that time period can be directly obtained. The above three aspects provide quantitative characteristics from three orthogonal dimensions: frequency domain structure, multi-scale complexity, and time domain rate of change. Furthermore, directly scanning and recording the locations and values ​​of local extrema exceeding a preset threshold in the time domain aims to anchor a set of clear, stable, and reproducible physical reference points for the entire system. These reference points are crucial for the lossless recovery of data at the receiving end, as they represent key, tamper-proof sampled values ​​in the original signal. For example, when analyzing impact test data from a vibration sensor, the system uses a sliding window to extract the data. For a window containing the initial impact phase, its time-domain difference average will suddenly increase, indicating a drastic change. Its short-time Fourier transform spectrum may show that energy rapidly concentrates from a broad distribution to a specific frequency band, and the principal component concentration feature captures this change. Its discrete wavelet transform coefficients at fine scales become active due to the transient characteristics of the impact, leading to a significant increase in Shannon entropy at that scale. Simultaneously, the peak points generated by the impact within this window are identified as local extrema, and their precise time and amplitude are recorded as feature point sampled values. All these dynamically calculated vectors and recorded sample values ​​together constitute a digital fingerprint describing this shock event.

[0040] This application performs short-time Fourier transform and discrete wavelet transform on the sliding window data in parallel, and comprehensively calculates multi-dimensional features such as spectral concentration, Shannon entropy, and rate of change, thereby comprehensively and accurately characterizing the time-frequency domain characteristics and transient details of high-frequency test signals. This solves the problem of insufficient descriptive ability of traditional single feature extraction methods, providing a richer and more reliable basis for subsequent adaptive compression decisions, thus improving the overall adaptability and efficiency of compression.

[0041] In some embodiments of this application, the dynamic selection of compression combination algorithm in step S2 specifically involves: inputting the multidimensional dynamic feature vector extracted in step S1, along with the current available network bandwidth and the target compression ratio, into a pre-trained classification and regression decision tree model; the classification and regression decision tree model first outputs a compression mode classification, which corresponds to a compression algorithm sequence framework, and then outputs a set of specific parameter configurations for each algorithm step in the compression algorithm sequence framework, thereby realizing the dynamic selection of compression combination algorithm based on the specific parameter configurations.

[0042] Understandably, a pre-trained classification and regression decision tree model is a machine learning model that combines the functions of classification and regression trees. After training with a large amount of historical data and corresponding optimal decisions, this model can be deployed without real-time online relearning. Compression mode classification is the first judgment task performed by this model. It maps the current input to a predefined, finite number of category labels, such as stationary efficient mode, transient fidelity mode, or periodic compression mode. Each compression mode classification label is associated with a compression algorithm sequence framework, a pre-designed template process that specifies the type and execution order of compression algorithms. For each algorithmic step in the framework, the model further performs a regression task, outputting a set of specific operating parameters, such as the order of predictive coding, the window size of dictionary coding, or the code table type of entropy coding. These specific parameter configurations allow the same algorithm framework to finely adapt to different data sub-characteristics.

[0043] In this embodiment, the principle behind this step lies in decomposing the complex problem of compression strategy selection into a hierarchical decision-making process: first, classifying to determine the main framework, then regressing to fine-tune parameters. Real-time inference is achieved through an efficient and interpretable tree model. The model first selects branches based on threshold judgments, starting from the root node, based on the input multidimensional dynamic feature vector, the current network bandwidth, and the target compression ratio, ultimately reaching a leaf node. This leaf node corresponds to a compression mode classification. Essentially, this involves selecting the most likely successful general-direction strategy framework based on the overall characteristics of the data, such as whether it is predominantly stationary or transient, whether the spectrum is concentrated, and external constraints such as bandwidth availability. The decision-making process does not end after determining the main framework. Within the same leaf node, the model also stores independent regression sub-models for each algorithm step to be configured under that framework. These sub-models also output continuous specific parameter values ​​based on the current complete input information. For example, after selecting a framework that includes predictive coding and dictionary coding, the model will output the order of the predictor and the sliding window size of the dictionary, respectively. This design achieves integrated and adaptive linkage between macro-strategy and micro-parameters. The entire decision-making process relies on offline training using a large amount of labeled data. The training data includes various data features, network conditions, and manually defined optimal compression strategies and parameter combinations. The goal of model learning is to approximate this mapping from complex conditions to optimal decisions. When used online, this pre-trained model can instantly complete inference, achieving real-time dynamic customization of the compression strategy. Assume the current input data features show smooth signal changes and a concentrated spectrum, while network bandwidth is relatively limited. Multidimensional dynamic feature vectors, bandwidth values, and the target compression ratio are input into the pre-trained classification-regression decision tree model. After internal judgment, the model first outputs a compression mode classification as a high compression ratio mode. This classification corresponds to a compression algorithm sequence framework, such as: "Preprocessing smoothing -> High-order linear predictive coding -> Run-length coding -> Static Huffman coding". Next, the model outputs specific parameters for each stage of this framework: recommending a mean filter window of a specific length for the preprocessing smoothing stage; recommending a higher order, such as 10th order, for the linear predictive coding stage; recommending a specific maximum run length for the run-length coding stage; and instructing the static Huffman coding stage to use a code table predefined for the statistical characteristics of stationary signals. Thus, the system dynamically selects and instantiates a combination of high-compression algorithms configured with specific parameters to adapt to scenarios with smooth data and limited bandwidth, striving to achieve smaller data volume with higher computational complexity. Conversely, if the data is rapidly changing and bandwidth is sufficient, the model selects a high-fidelity framework and configures it with edge-preserving preprocessing, low-order or nonlinear prediction, and coding parameters based on a dynamic dictionary.

[0044] This application employs a pre-trained classification and regression decision tree model, using multi-dimensional features, network bandwidth, and the target compression ratio as inputs to achieve integrated dynamic selection from compression mode to specific parameters. This solves the problem of rigid, manually pre-configured compression strategies being unable to adapt to changing data patterns, realizing intelligent and automated generation of compression strategies, and significantly improving the compression process's adaptability to different data characteristics and overall compression performance.

[0045] In some embodiments of this application, step S2 further includes online fine-tuning of the decision tree model. During the compression process, the actual compression ratio and compression speed are collected in real time. When the deviation between the actual compression ratio and the target compression ratio continues to exceed the set range, or the compression speed is lower than expected, the decision threshold or output value of the corresponding node of the current decision tree model is corrected online.

[0046] Understandably, online fine-tuning refers to the process of making small-scale, incremental adjustments to the model's internal parameters based on newly generated data and feedback after the machine learning model has completed its initial training and been deployed to the actual operating environment, without interrupting its service. This differs from offline retraining, and its goal is to enable the model to quickly adapt to local changes in the environment or data distribution without consuming significant computational resources for global updates. The decision threshold is the key value used to compare with the input feature value at each internal decision node of a classification / regression decision tree model. When the feature value is less than or equal to the threshold, the data flows to one branch; otherwise, it flows to another. Online correction means dynamically raising or lowering these thresholds based on real-time feedback during system operation. In regression tasks, the output value typically refers to the specific numerical value predicted by the model; in this context, it specifically refers to the predicted value relative to the compression algorithm parameter configuration, such as the order of the prediction code or the dictionary size. Online correction of these output values ​​is a calibrative adjustment based on actual performance feedback.

[0047] The principle behind this step in this embodiment is to introduce a dynamic adaptation and self-optimization closed loop into the static pre-trained decision tree model. Its core lies in establishing a model parameter fine-tuning mechanism based on real-time performance feedback to address unknown data patterns or environmental fluctuations not fully covered during the pre-training phase. While performing the compression task, the system continuously monitors two key performance indicators: the actual achieved compression ratio and the compression speed. These measured results and the target expectation constitute a direct feedback signal. The system has a reasonable tolerance range for deviations. When the deviation between the actual compression ratio and the target value exceeds the set range multiple times consecutively, or the compression speed is consistently lower than the preset minimum expectation, the system determines that the current decision may not be applicable to the actual data segment being processed. At this point, an online fine-tuning process is triggered. The system associates the multi-dimensional dynamic feature vectors and network bandwidth corresponding to the currently processed data with the measured results that lead to substandard performance. The system locates the key judgment nodes (or nodes) in the decision tree that cause the current decision path. The fine-tuning algorithm analyzes which feature's threshold judgment at which node may be unreasonable, or whether the final output parameter value needs adjustment. For example, it might discover that for data with a newly emerging combination of features, the original model tends to choose high-order predictive encoding, but tests show that low-order encoding is faster and achieves the desired compression ratio. Therefore, the system will slightly shift the decision thresholds of relevant nodes or directly update the output parameter values ​​using a weighted average. This fine-tuning is incremental and local, aiming to quickly correct obvious local misjudgments without disrupting the model's overall knowledge structure. The corrected model will then be immediately applied to decisions on subsequent data, forming a continuous learning loop of "decision-execution-evaluation-correction," thereby significantly improving the system's robustness and environmental adaptability in long-term operation.

[0048] This application introduces an online fine-tuning mechanism for the decision model based on real-time compression performance feedback, enabling the system to continuously learn and optimize during operation. This addresses the issue of static models potentially failing when faced with changes in local characteristics of the data stream or environmental fluctuations, enhancing the system's robustness and adaptability, and ensuring that compression performance remains stable near its optimal state over the long term.

[0049] In some embodiments of this application, step S3 processes the high-frequency test data according to the compression and combination algorithm, specifically including: dividing the high-frequency test data into overlapping frames and non-overlapping frames according to the data continuity requirements of the selected compression and combination algorithm; and labeling each frame of data with a type label based on spectral characteristics, complexity characteristics, and data change rate characteristics. The type labels include stationary frames, transient frames, and periodic frames.

[0050] It's understandable that overlapping and non-overlapping frames represent two different methods of data framing. Non-overlapping frames extract continuous data streams sequentially at fixed lengths, with adjacent frames joined end-to-end without shared data. Overlapping frames, on the other hand, allow for some shared data samples between adjacent frames, meaning the starting point of the next frame lies within the previous frame. Overlapping framing helps smooth out boundary effects caused by framing, making it particularly suitable for frequency domain analysis with high continuity requirements or processing algorithms that need to preserve boundaries. Type labels are semantic classification identifiers assigned to each frame of data. Based on the quantization characteristics calculated from the frame's data, they categorize data frames into several typical types, such as stationary frames, transient frames, and periodic frames. Stationary frames indicate slow signal changes and stable statistical characteristics within that time period; transient frames indicate rapid abrupt changes or shocks in the signal; and periodic frames indicate that the signal exhibits a clear repetitive oscillation pattern.

[0051] In this embodiment, the principle of this step is to perform structured segmentation and classification of the original continuous data based on the characteristics of the compression algorithm and the characteristics of the data itself, preparing highly organized input units for subsequent differentiated compression processing, thereby achieving a precise improvement in compression efficiency. First, the framing strategy is selected based on the inherent requirements of the chosen compression combination algorithm. For example, some block-transformation-based compression algorithms require strictly independent data blocks, so they use non-overlapping frames; while some context-modeling-based algorithms, in order to avoid losing relevance at frame boundaries, may prefer to use a frame structure with a certain degree of overlap to ensure the continuity of the data stream context information. This step is a preliminary physical organization of the original data to meet the basic requirements of different algorithms for the input format. Second, based on framing, the system uses the feature vector extracted for the entire data stream in step S1 (specifically to the time window corresponding to each frame) to intelligently label the type of each frame. The labeling is based on a combination of spectral features, complexity features, and data change rate features. For example, if a frame of data has a very low data change rate feature value, a spectral feature showing highly concentrated energy, and a low complexity feature value, it is likely to be labeled as a stationary frame. If the rate of change of data suddenly spikes while the complexity of the wavelet domain increases significantly, it is likely to be labeled as a transient frame. If the spectral features show a clear discrete spectral structure, it may be labeled as a periodic frame. This preprocessing of framing and classification plays a crucial role in bridging the gap between the preceding and following steps. It ensures that subsequent compression execution units no longer face a homogeneous data stream, but rather a series of data frames with clearly identified characteristics. This allows the system to use tailored processing parameters or even sub-processes for different types of data frames in subsequent steps, thereby enabling more refined adaptation of the compression combination algorithm at the microscopic level. Suppose the system currently selects a compression algorithm combination that is sensitive to continuity and based on long-term prediction, and therefore decides to use an overlapping framing method with a 50% overlap rate. A vibration signal from rotating machinery is input. After the first frame is captured, the system retrieves the feature vector corresponding to that frame: its rate of change feature value is very low, the spectral features show that the energy is mainly concentrated on the fundamental frequency and its harmonics corresponding to the rotational speed, and the complexity feature value is moderate. The system labels it as a periodic frame according to predefined classification rules. The second frame (overlapping 50% with the first) is captured, at which point a brief impact pulse suddenly enters the signal. The feature vector of this frame shows a spike in the rate of change characteristic, a broadened spectral characteristic, and a sharp increase in complexity. The system labels this as a transient frame. In the third frame, the impact has passed, and the signal returns to near steady state. Its rate of change characteristic returns to a low level, the spectral characteristic becomes more concentrated again, and the complexity decreases. The system labels this as a stationary frame. Through this processing, the original vibration signal stream is transformed into a series of frames with clearly defined type labels: periodic frames, transient frames, stationary frames, and so on.When subsequent compression modules receive these frames, they know that specialized harmonic analysis and periodic predictive coding can be used for periodic frames, a high-fidelity-priority fast coding strategy should be adopted for transient frames, and smooth coding with a high compression ratio can be attempted for stationary frames. This type-based differentiated processing is the key to improving overall compression efficiency.

[0052] This application provides structured data units and clear processing guidance by framing and labeling the raw data according to the requirements of the compression algorithm and data characteristics. This solves the problem of difficulty in implementing targeted optimization strategies when processing data streams as a whole, achieves preliminary data classification and management, and lays the foundation for implementing differentiated and efficient preprocessing.

[0053] In some embodiments of this application, step S3 further includes a differential preprocessing step corresponding to the frame type label, specifically including: for data labeled as stationary frames, a moving average filter is used for smoothing to reduce random noise; for data labeled as transient frames, an edge-preserving filter is used for noise reduction while maintaining the edges of abrupt changes, and the start and end points of abrupt changes are marked with high precision; for data labeled as periodic frames, a base period estimation and phase alignment operation based on the autocorrelation function is performed to improve the efficiency of the periodic compression algorithm.

[0054] Understandably, the moving average filter is a fundamental method for smoothing time-domain signals. Its principle is to replace the original value of each data point in the signal sequence with the arithmetic mean of itself and several neighboring data points. This method effectively suppresses high-frequency random fluctuations or noise, but at the cost of potentially blurring the edges of rapid signal changes. Edge-preserving filters are a more advanced filtering technique designed to smooth noise while preserving as many sharp edges or abrupt changes in the signal as possible. They typically analyze the statistical characteristics or gradient information of local regions and adaptively adjust the filter weights, thus implementing strong smoothing in flat areas and weak or no smoothing near edges. High-precision labeling here specifically refers to the precise calibration of the start and end times of signal abrupt events beyond the original sampling interval, possibly achieved by interpolation algorithms to locate extrema or zero-crossing points between sampling points. The autocorrelation function is a mathematical tool used to measure the similarity between a signal and its time-shifted replica. By finding the peak position of the autocorrelation function, the fundamental repetition period hidden in the main signal can be effectively estimated. Fundamental period estimation is the determination of the shortest repetition unit length in the periodic pattern of the signal through analysis of the autocorrelation function and other calculations. Phase alignment refers to shifting different periodic segments of a periodic signal on the time axis so that their starting points or specific feature points are aligned with the same reference phase, thereby eliminating data inconsistencies caused by random phase start.

[0055] In this embodiment, the principle of this step lies in applying targeted signal shaping and enhancement operations based on the fundamental characteristics of the identified data frames. The aim is to optimize the data's form, making it more suitable for the selected compression algorithm, thereby enhancing compression potential from the source. For stable frames, the main characteristic is that background noise may mask small, useful fluctuations, but the overall signal trend is smooth. Using a moving average filter for smoothing can filter out high-frequency random noise while maintaining the overall trend. Since noise itself is unpredictable and difficult-to-compress redundant information, eliminating it directly reduces the data entropy that needs to be encoded, providing a more regular input for subsequent compression algorithms, thus improving the compression ratio. For transient frames, their value lies precisely in the edge information of abrupt changes; any blurring will lose key features. Therefore, using an edge-preserving filter for noise reduction removes noise that may exist inside and outside the abrupt change region while protecting these important edge structures from damage. Furthermore, high-precision marking of the start and end points of abrupt changes adds accurate metadata indexes to these key events. These markers themselves contain very little data, but they provide strong contextual clues for the compression algorithm, enabling it to employ special local coding strategies around these points to achieve efficient compression while maintaining high fidelity. For periodic frames, the biggest obstacle to compression efficiency lies in the imperfect alignment of the periods and the randomness of the phases. Estimating the base period based on the autocorrelation function can accurately identify the length of the basic unit of signal repetition. The subsequent phase alignment operation aligns each periodic waveform to a consistent starting point on the time axis. After alignment, the signal transforms from a quasi-periodic signal with disordered phases into a strictly aligned periodic signal. This allows subsequent predictive coding or specialized sinusoidal coding algorithms to model and predict the signal extremely accurately, requiring only the encoding of minimal residual information, thus achieving extremely high compression efficiency. Assume a sound test signal contains: a steady white background noise, a brief metallic knocking sound, and a steady motor hum. The system has labeled them as steady frames, transient frames, and periodic frames, respectively. For the steady frame (white background noise), the system applies a 5-point moving average filter. The original sample sequence, such as [10, 12, 8, 11, 9, 13, 7, ...], is smoothed to [10, 10, 10.2, 9.6, 10.2, 10.6, 9.8, ...]. Random fluctuations are suppressed, and the sequence becomes smoother. For transient frames (metallic knocking sounds), the system applies an edge-preserving filter. The sharp edges of the original impact pulses are completely preserved, such as [50, 52, 180, 175, 50, 48, ...]. After filtering, it may become [50, 51, 178, 173, 50, 49, ...]. The edges are still steep, but the minor perturbations of individual samples are corrected.Simultaneously, the system accurately calculated that the abrupt change started near the third sampling point within the frame and ended near the fifth sampling point, recording these positions. For a periodic frame (motor humming sound), the system calculated its autocorrelation function, finding that the peak interval was 30 sampling points, thus determining the base period to be 30. Subsequently, the system cut and aligned the data of this frame, along with the data of adjacent periodic frames, with a period of 30 points, ensuring that the positive peak of each periodic waveform was aligned to the same reference position. After alignment, the originally phase-interleaved periodic waveform became uniform.

[0056] This application employs specific preprocessing operations for different frame types, such as stationary, transient, and periodic frames, including moving average smoothing, edge-preserving noise reduction and labeling, and period estimation and phase alignment. This solves the problem that traditional single preprocessing methods may damage the characteristics of specific data types (such as blurring edges or destroying periodicity). It can remove noise interference while preserving or enhancing data structures that are beneficial to subsequent compression to the greatest extent, thereby significantly improving the compression potential of various types of data.

[0057] In some embodiments of this application, the intermediate correction step in step S4 specifically includes: performing statistical characteristic analysis on the output data block of the previous algorithm, calculating its data distribution histogram, mean, and variance; comparing the statistical characteristics with the ideal input data distribution range preset for the next algorithm in the currently selected compression combination algorithm; if there is a deviation, transforming the data block through a numerical mapping function to adjust its statistical characteristics to the preset ideal range, and then inputting the adjusted data block into the next algorithm.

[0058] Understandably, a data distribution histogram is a graphical representation that divides the data value range into several continuous intervals and counts the number of data points falling into each interval. It intuitively shows the concentration and dispersion of values ​​in the entire data block. The mean is the arithmetic mean of all data points, representing the central location of the data distribution. The variance is the average of the squares of the differences between each data point and the mean; it measures the degree of data fluctuation around the mean, i.e., the dispersion of the data. The ideal input data distribution range is a recommended specification regarding the statistical characteristics of its input data (such as the mean range, upper limit of variance, and distribution shape), pre-defined based on the design principles of a specific compression algorithm and extensive experiments to ensure the algorithm's efficient and stable operation. A numerical mapping function is a function used to convert input values ​​into output values ​​according to specific mathematical rules. In the bias correction stage, it specifically refers to the transformation rules designed to adjust the data block from the current statistical distribution to the target distribution, such as linear scaling, translation, or nonlinear functions.

[0059] In this embodiment, the principle of this step lies in identifying and actively correcting the statistical mismatch problem between data stages in a multi-stage compression algorithm pipeline. Its core is to use an adaptive data shaper to ensure that each algorithm receives input data close to its optimal operating state, thereby guaranteeing the stable and efficient operation of the entire pipeline. Compression algorithms are typically designed for data with specific statistical characteristics. For example, a run-length encoding is most efficient for continuous sequences of identical values, while an entropy encoding has the highest compression rate for data with highly concentrated symbol probability distributions. When multiple algorithms are cascaded, the statistical characteristics (such as numerical range and distribution shape) of the output data blocks of the preceding algorithm may change due to the nonlinear effects of the preceding processing, thus deviating from the ideal input range preset by the subsequent algorithm. This mismatch directly leads to a decrease in the performance of the subsequent algorithm and may even cause errors. The intermediate correction stage provides a dynamic adapter for this problem. It first performs a rapid statistical characteristic analysis on the output data blocks of the preceding stage, calculates its key descriptive indicators, such as mean and variance, and understands its distribution shape through histograms. Subsequently, the system compares this measured statistical characteristic with the ideal input data distribution range preset for the subsequent algorithm. Ideally, the input data distribution range might be defined as having a mean close to zero, a variance within a certain interval, and an approximately symmetrical distribution. Once a significant deviation is detected, the correction stage initiates a numerical mapping function to transform the entire data block. For example, if the measured mean deviates from zero, a translation operation is performed to bring the mean to zero; if the measured variance is too large, all data points are linearly scaled to reduce the variance to the target interval; if the distribution is severely skewed, a logarithmic or power-law transformation might be applied to make it closer to a symmetrical distribution. After this transformation, the basic statistical form of the data block is adjusted to the comfort zone of subsequent algorithms before being fed into them for processing. For example, a compression-combination algorithm uses linear predictive coding in the first stage and Huffman entropy coding in the second stage. The first-stage predictive coding outputs the prediction residuals. Ideally, efficient entropy coding expects the input residual data to be highly concentrated, preferably with a small variance and a mean close to zero. However, when processing a specific set of data, statistical analysis of the residual data block output by the predictive coding shows a mean of 5, a variance of 100, and a relatively dispersed distribution as shown in the histogram. The ideal input range preset for Huffman coding is a mean of [-1, 1] and a variance of [1, 50]. An intermediate correction step is triggered. First, it subtracts 5 from all data points to correct the mean to 0. Next, it calculates that the ratio of the current variance of 100 to the ideal upper limit of 50 is approximately the square root of 2. Then, it performs a linear scaling operation (scaling mapping) on ​​all data points, dividing by 1.414. After translation and scaling, the new data block has a mean of approximately 0 and a variance of approximately 50. The distribution shape remains unchanged but is compressed overall, falling within the ideal input range for Huffman coding.The adjusted data block is fed into a Huffman encoder. Due to the more concentrated numerical distribution, the encoder can construct a more efficient code table, thus achieving a higher compression ratio than directly encoding the original residual data block. This example clearly demonstrates how the bias correction stage can mitigate the efficiency loss caused by the mismatch of input characteristics between stages through simple statistical transformations.

[0060] This application introduces a correction mechanism based on statistical characteristic analysis between compression pipeline stages. Through numerical mapping, it adjusts the output data of the preceding stage to the ideal input range of the subsequent algorithm. This solves the problem that when multiple compression algorithms are cascaded, the output of the preceding stage may deviate from the efficient operating range of the subsequent algorithm, leading to decreased compression efficiency or even distortion. It ensures that each stage of the algorithm operates at its optimal state, improving the stability and output quality of the overall compression pipeline.

[0061] In some embodiments of this application, the transmission state matrix in step S5 is a three-dimensional matrix, whose three dimensions represent different transmission priority levels of data packets, different alternative network link identifiers, and a link quality score calculated by combining real-time network latency, packet loss rate, and estimated throughput. According to the transmission priority of the data packets, the corresponding two-dimensional sub-matrix is ​​selected in the matrix, and then a preset algorithm is used to select the link with the best link quality score and not currently under full load for each data packet in the two-dimensional sub-matrix for binding.

[0062] Understandably, the transmission state matrix in this scheme is constructed as a three-dimensional logical structure to systematically organize all the decision-making information required for transmission scheduling. The three dimensions correspond to three types of key parameters: transmission priority level represents a discrete level based on the importance of the data content, such as high, medium, and low levels; alternative network link identifiers are unique codes used to distinguish different available physical or logical transmission channels, such as Wi-Fi networks, 5G mobile networks, and wired Ethernet; the link quality score is a single quantitative score representing the current transmission performance of the link, calculated by a comprehensive evaluation function that integrates real-time monitored network latency, packet loss rate, and estimated throughput. A higher score generally indicates better link quality. The two-dimensional submatrix is ​​a slice extracted from the three-dimensional matrix according to a specific dimension (here, priority level). In this specific scenario, it is a table with the horizontal axis representing alternative network links and the vertical axis representing the link quality score. The preset algorithm refers to the pre-defined calculation rules for completing the specific task of link selection. It can be a simple greedy selection algorithm or a more complex optimization algorithm that considers load balancing.

[0063] The principle behind this step in this embodiment lies in using a structured, multi-dimensional state model to uniformly represent and efficiently match differentiated data demands with dynamically changing network resources, thereby achieving optimal network resource scheduling and ensuring the fast and reliable transmission of high-priority data. Its core is the construction and maintenance of a three-dimensional transmission state matrix. The first dimension (priority level) distinguishes the differences in data demands; data with different priorities have different requirements for transmission speed and reliability. The second dimension (link identifier) ​​enumerates all available transmission resource options. The third dimension (link quality score) dynamically reflects the real-time supply capacity or health status of each resource. When a transmission link needs to be allocated for a data packet, the scheduler first locks the corresponding two-dimensional sub-matrix in the three-dimensional matrix based on the data packet's own transmission priority. This sub-matrix clearly shows the quality scores of all available links for the current priority data. This simplifies the problem to an optimal resource selection problem for a specific demand. Subsequently, the scheduler calls a preset algorithm to make a decision in this two-dimensional sub-matrix. The algorithm not only needs to find the link with the highest link quality score but also must consider the current load of that link to avoid congesting too many data packets onto a single high-quality link. Therefore, algorithms typically trade off link quality against link load. For example, they might select the first available link with a higher quality score and a current queue of packets not exceeding a threshold, or employ a weighted round-robin strategy. Once selected, the packet is bound to that link and ready for transmission. This method achieves fine-grained, demand-aware dynamic routing. High-priority data can preferentially occupy the link with the best quality, while low-priority data utilizes the remaining link capacity, achieving a reasonable allocation and efficient use of network bandwidth resources overall. Assume the system defines three transmission priorities: high, medium, and low. Currently, there are three available links: Wi-Fi, 5G, and wired network. The system calculates the comprehensive quality scores of the three links in real time: Wi-Fi (85 points), 5G (70 points), and wired network (95 points). This information constitutes a three-dimensional transmission state matrix. Now, a compressed data packet P1 marked as high priority needs to be sent. The scheduler selects the two-dimensional submatrix corresponding to the high priority in the matrix, which shows that for high-priority data, the quality scores of the three links are 85, 70, and 95. The pre-defined algorithm (e.g., a simple greedy algorithm that selects the highest-scoring link that is not fully loaded) searches in this submatrix and finds that the wired network has the highest score (95 points) and its current sending queue is not full. Therefore, the algorithm binds data packet P1 to the wired network link. Next, a data packet P2 marked as medium priority needs to be sent. The scheduler switches to the two-dimensional submatrix corresponding to the medium priority (although the link score values ​​may be the same as those in higher layers, in more complex implementations, different priorities may assign different weights to the same link score; here, for simplicity, they are treated as the same).The algorithm ran again and found that although the wired network had the highest score, its load was close to saturation after sending P1. Therefore, the algorithm skipped it according to the rules and selected the Wi-Fi link (85 points) with the second highest score and lighter load to bind with P2. Through this mechanism, the high-priority data packet P1 is transmitted quickly through the best quality wired network, while the medium-priority data packet P2 is transmitted through the good quality Wi-Fi network, which ensures both fast transmission of critical data and balanced link load.

[0064] This application achieves refined management of data packet transmission by constructing a three-dimensional transmission state matrix that integrates transmission priority, link identification, and link quality score, and by performing optimal link matching based on a priority selection submatrix. This solves the problem in traditional transmission scheduling of the difficulty in coordinating differences in data importance and the variability of network link states, and can intelligently guide important data to high-quality links, thereby optimizing overall transmission efficiency and reliability.

[0065] In some embodiments of this application, the process of adding redundant forward error correction codes in step S5 is dynamically associated with link allocation. First, the system maintains a lookup table of link packet loss rate and required error correction redundancy. After allocating a specific link to a data packet, the system immediately queries the real-time packet loss rate of that link and dynamically determines the type and redundancy of the error correction code to be added based on the lookup table. For high-priority data packets allocated to links with high packet loss rates, a coding method with strong error correction capability but high overhead is adopted. For low-priority data packets allocated to high-quality links, lightweight verification or even no error correction code is added.

[0066] Understandably, the table mapping link packet loss rate to required error correction redundancy is a pre-established mapping relationship through experiments or theoretical models. It specifies the recommended range of additional check data to be added in the forward error correction stage, along with recommended coding types, to achieve the desired end-to-end reliability when the real-time packet loss rate of the data transmission path is within a certain range. Error correction code type refers to the specific coding scheme implementing forward error correction, such as Reed-Solomon codes, convolutional codes, or Turbo codes. Each type has its own characteristics in terms of error correction capability, computational complexity, and additional overhead. Redundancy specifically refers to the proportion of check data added for error correction relative to the amount of original information data, usually expressed as a percentage or ratio. Coding schemes with strong error correction capability but high overhead refer to those that can tolerate high bit error rates and correct multiple burst or random errors, but require more check bits, leading to a decrease in effective data transmission rate. Lightweight check usually refers to check mechanisms that only have the ability to detect errors or can only correct a very small number of errors, such as cyclic redundancy check, which has fast computation speed and few additional bits.

[0067] In this embodiment, the principle behind this step is to transform the error protection strategy from a static, uniform configuration into an adaptive process closely coupled with real-time network transmission conditions and the value of the data itself. Its core objective is to minimize unnecessary error correction overhead while meeting data reliability requirements, thereby optimizing overall effective network throughput. The system maintains a key lookup table that establishes a quantitative relationship between the current packet loss rate of a link and the recommended error correction configuration. This lookup table may be based on theoretical calculations of the channel model or obtained through statistical analysis of numerous historical transmission experiments. The underlying logic is: a higher packet loss rate on a link indicates a more severe transmission environment, requiring stronger error correction capabilities and higher redundancy protection to ensure correct data delivery; conversely, for stable, high-quality links, the protection strength can be significantly reduced to save bandwidth. The decision-making process does not end after a data packet is dynamically assigned to a specific link according to its priority. The system immediately queries the latest real-time packet loss rate for the link and, based on the aforementioned lookup table, quickly determines the specific type of error correction code to be used (e.g., whether to choose a code resistant to burst errors or random errors) and redundancy parameters (e.g., whether to add 20% or 50% checksum data). This process is dynamic and precise, ensuring that each data packet receives just the right amount of protection. More importantly, this mechanism is linked to data packet priority. For high-priority data packets unfortunately assigned to high-loss-rate links, the system tends to select codes with extremely strong error correction capabilities (e.g., capable of correcting multiple errors), even if the overhead is significant, because the value of the data outweighs the bandwidth cost. For low-priority data packets fortunately assigned to high-quality links, the system may only add lightweight checksums for error detection, or even omit any error correction codes without affecting overall reliability, reserving the saved bandwidth resources for more important data. This allows valuable network resources to be intelligently and differentiated based on the importance of the data and the reliability of the link. Suppose the system lookup table specifies: links with a packet loss rate greater than 5% are recommended to use Reed-Solomon coding with 30% redundancy; links with a packet loss rate between 1% and 5% are recommended to use convolutional coding with 15% redundancy; and links with a packet loss rate less than 1% only require cyclic redundancy check (CRC) for error detection. A high-priority characteristic data packet P_high is assigned by the scheduler to a cellular mobile network link with a current packet loss rate of 8%. The system immediately checks the lookup table and finds that an 8% packet loss rate falls within the "greater than 5%" range. Therefore, the system decides to add 30% Reed-Solomon checksum to the data payload of P_high, forming strong protection before transmission. Simultaneously, a low-priority background data packet P_low is assigned to a stable Wi-Fi link with a current packet loss rate of only 0.3%. The system checks the lookup table and finds that 0.3% falls within the "less than 1%" range.Therefore, the system only adds a short cyclic redundancy check (CRC) code to P_low, resulting in minimal overhead. If the network is extremely stable at this time, the system policy may even allow sending the original compressed packet directly without adding any error correction code for such data. In this way, high-priority data is protected in poor-quality channels, while low-priority data is transmitted in high-quality channels. The system as a whole maximizes the efficiency of network bandwidth resource utilization while ensuring the reliability of critical data.

[0068] This application dynamically correlates the redundancy of forward error correction codes with the real-time packet loss rate of the specific link allocated to the data packet, establishing a differentiated error protection strategy. This solves the problem that fixed error correction schemes either provide insufficient protection leading to retransmissions or over-protect and waste bandwidth in variable network environments, achieving an adaptive balance between network resources and reliability requirements, maximizing effective data transmission efficiency while ensuring the reliability of critical data.

[0069] In some embodiments of this application, the receiving end first uses redundant forward error correction codes to detect and correct errors in the received data packets. Correctable errors are directly repaired, while errors exceeding the error correction capability are marked as suspicious packets. Based on the corrected data and compression strategy information, preliminary decompression is performed to obtain the reconstructed data stream to be verified. According to the feature point location index information associated with the sending end and the compressed data stream, each feature point is accurately located in the reconstructed data stream to be verified. The actual sampled values ​​at the corresponding positions in the reconstructed data stream are extracted to form a sequence of reconstructed feature point sampled values. The sequence of reconstructed feature point sampled values ​​is compared with the original feature point sampled value sequence synchronized by the sending end. The comparison dimensions include numerical absolute error, local waveform similarity, and statistical distribution consistency. Only when the comparison results of all dimensions meet their respective preset tolerance thresholds is the transmission and reconstruction process of the batch of data determined to be lossless. If any dimension comparison fails, a retransmission request for the suspicious data packet or the corresponding data segment is triggered.

[0070] Understandably, suspicious packets refer to data packets whose number or pattern of errors exceeds the corrective capability of the error correction code design during forward error correction processing at the receiving end. Although these packets undergo error correction attempts, the correctness of their content cannot be guaranteed, and therefore they are specially marked by the system for further special processing. Feature point location index information is a set of metadata generated by the sending end in step S1 and synchronized to the receiving end along with the compression strategy information. It explicitly records the specific position number or timestamp of each marked feature point in the original data stream. The reconstructed feature point sample value sequence is a sequence formed by the receiving end after initial decompression, based on the received location index information, extracting the actual values ​​from the corresponding positions in the reconstructed data stream to be verified, and arranging them sequentially. The absolute numerical error is the absolute value of the difference between the reconstructed feature point sample value and the original feature point sample value during comparison, used to measure the accuracy of single-point data recovery. Local waveform similarity is a measurement method that is not limited to a single sampling point. Instead, it selects a segment of data waveforms surrounding a feature point and evaluates the consistency of the reconstructed waveform with the original waveform in terms of local morphology by calculating correlation coefficients or dynamic time warping distances. Statistical distribution consistency, on the other hand, is at a more macroscopic level. It calculates and compares the overall statistical characteristics of the reconstructed data segment and its corresponding original data segment, such as mean, variance, skewness, and kurtosis, to determine whether their probability distributions are the same or extremely close. Tolerance thresholds are preset upper limits of acceptable error or lower limits of similarity for each of the above comparison dimensions. Only when the comparison result is better than the threshold is it considered acceptable. A retransmission request is a control signal initiated by the receiving end to the sending end after verification failure, requesting that specific data packets marked as suspicious or the original data segment corresponding to the verification failure be recompressed and retransmitted.

[0071] In this embodiment, the principle behind this step lies in constructing an end-to-end deep verification and closed-loop guarantee mechanism that transcends traditional communication link-layer verification and is based on the key characteristics of the data content itself. This step is executed in stages. The first stage is link-layer protection and preliminary repair, using forward error correction codes to attempt to correct bit errors introduced during transmission, dividing data packets into repaired trusted packets and suspicious packets, laying a preliminary foundation for subsequent processing. The second stage is preliminary reconstruction and precise positioning. The system combines the error-corrected data with synchronized complete compression strategy information to perform a reverse decompression process, restoring the structured data stream. Simultaneously, using the feature point location index synchronized at the sending end, much like using a precise coordinate map, the system unambiguously locates the accurate position of each feature point in the reconstructed data stream. The third stage is multi-dimensional feature comparison and final decision. The system does not simply compare the entire data stream, but focuses on the feature points that best represent the essence of the data and are sensitive to errors. The comparison unfolds across three complementary dimensions: numerical absolute error ensures that the value of each key sampling point is accurately restored; local waveform similarity ensures that the signal morphology (such as pulse shape and peak profile) near the feature points is not distorted; and statistical distribution consistency ensures that the overall characteristics of the data segment (such as noise level and signal energy distribution) remain consistent from a global perspective. This multi-dimensional comparison combining points, lines, and surfaces constitutes an extremely rigorous verification network. Only when all dimensions meet the preset stringent tolerance thresholds does the system ultimately determine that the batch of data has achieved true lossless recovery. The fourth stage is closed-loop error correction. If any dimension comparison fails, it means that lossless performance has been compromised. At this point, the system does not simply discard the data, but instead associates it with previously marked suspicious packet information or locates the specific data segment based on the failed feature points, and initiates a precise retransmission request to the sender. This forms a negative feedback loop from verification failure to source retransmission, ultimately ensuring that the receiver always obtains data that has passed rigorous lossless verification. Suppose the transmitter sends a high-frequency test data segment containing a sharp pulse. The pulse peak is marked as a feature point, with its location index at the 1024th sampling point. The original sampling value is 5.0 volts. Upon receiving the data, the receiver first performs error correction decoding, successfully correcting a few random bit errors, and all data packets are marked as trustworthy. Next, a synchronous compression strategy is used for decompression, initially reconstructing the data stream. Based on the received index, the system locates the feature point near the 1024th sampling point. Then, a three-dimensional comparison is performed: First, the reconstructed value of this point is extracted, and the absolute error with the original value of 5.0 volts is calculated. It is found to be 0.01 volts, below the threshold of 0.05 volts, thus passing. Second, local waveforms of 10 sampling points before and after this point are extracted, and the correlation coefficient with the original local waveform is calculated. It reaches 0.995, above the threshold of 0.99, thus passing. Third, calculate the reconstructed data variance of the entire data segment containing the pulse (e.g., points 1000 to 1100), compare it with the original variance synchronized at the transmitting end, and if the relative error is 0.5%, which is below the 1% threshold, it passes.Since all dimensions passed, the system determines that the transmission and reconstruction were lossless. Conversely, if the pulse is deformed during compression or transmission, resulting in a local waveform correlation coefficient of only 0.98, the verification fails. The system will trace the data segment containing the feature point and request the sender to retransmit the compressed packet of the original data segment until the verification passes.

[0072] This application provides a complete lossless verification process, from error correction and recovery, initial decompression, precise feature point localization and extraction to multi-dimensional comparison. This solves the problems that a single verification method may not be able to detect all types of distortion, and that traditional methods are difficult to verify the consistency of data content features. Through a rigorous, multi-layered verification mechanism, it ensures that data is only deemed lossless when both content and features are strictly consistent, greatly improving the reliability and authority of data integrity verification.

[0073] See Figure 2 A second aspect of this application provides a system for adaptive compression and lossless fast transmission of high-frequency test data, comprising: a dynamic feature extraction module 21, used to perform real-time time-frequency domain analysis on the input high-frequency test data, extract multi-dimensional dynamic feature vectors, and mark feature point sampling values; a compression decision module 22, used to dynamically select a compression combination algorithm based on the extracted multi-dimensional dynamic feature vectors, combined with the currently available network bandwidth and the target compression ratio; a data preprocessing module 23, used to process the high-frequency test data according to the compression combination algorithm; and a multi-level compression execution module 24, used to execute the compression combination algorithm and convert the previous level algorithm into a single data processing module. The output is used as the input of the next-level algorithm, and an intermediate correction step is inserted between adjacent algorithms to adjust the input parameters of the next-level algorithm to obtain the compressed data packet; the transmission scheduling and protection module 25 is used to construct a transmission state matrix based on transmission priority, network delay and packet loss rate, dynamically allocate the compressed data packet to different network links based on the transmission state matrix, and add redundant forward error correction codes; the lossless verification and reconstruction module 26 is used to perform decompression in reverse after receiving the compressed data packet on different network links, and realize lossless verification of high-frequency test data by comparing the feature point sampling values ​​with the redundant forward error correction codes.

[0074] See Figure 3 , Figure 3 This is a schematic block diagram of an electronic device provided according to an embodiment of this application. Figure 3The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to execute the aforementioned adaptive compression and lossless fast transmission method for high-frequency test data.

[0075] It should be understood that, in the embodiments of this application, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0076] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0077] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store device type information.

[0078] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this application can execute the implementation methods described in any embodiment of the adaptive compression and lossless fast transmission method for high-frequency test data provided in the embodiments of this application, or they can execute the implementation methods of the electronic devices described in the embodiments of this application, which will not be repeated here.

[0079] In another embodiment of this application, an electronic device is provided. The electronic device stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the adaptive compression and lossless fast transmission method for high-frequency test data described in the above embodiments. Alternatively, the computer program can instruct related hardware to complete the process. The computer program can be stored in an electronic device, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable media can include any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0080] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0081] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0082] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0083] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.

[0084] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0085] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0086] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for adaptive compression and lossless fast transmission of high-frequency test data, characterized in that, Includes the following steps: S1. Perform real-time time-frequency domain analysis on the input high-frequency test data, extract multi-dimensional dynamic feature vectors, and mark the feature point sampling values; S2. Based on the extracted multidimensional dynamic feature vector, and combined with the current available network bandwidth and the target compression ratio, dynamically select a compression combination algorithm; S3. Process the high-frequency test data according to the compression and combination algorithm; S4. Execute the compression and combination algorithm, and use the output of the previous algorithm as the input of the next algorithm. Insert an intermediate correction step between adjacent algorithms to adjust the input parameters of the next algorithm to obtain the compressed data packet. S5. Construct a transmission state matrix based on transmission priority, network latency and packet loss rate, dynamically allocate the compressed data packets to different network links based on the transmission state matrix, and add redundant forward error correction codes to achieve lossless fast transmission of the high-frequency test data. S6. After the compressed data packet is received on the different network links, decompression is performed in reverse, and lossless verification of the high-frequency test data is achieved by comparing the feature point sampling value with the redundant forward error correction code.

2. The method according to claim 1, characterized in that, In step S1, the real-time time-frequency domain analysis includes capturing the data stream through a sliding window, performing short-time Fourier transform and discrete wavelet transform on the data within each window in parallel; calculating the concentration of signal energy on the first N principal components from the short-time Fourier transform results as a spectral feature, calculating Shannon entropy from the multi-scale coefficients of the discrete wavelet transform as a complexity feature, and calculating the average value of the absolute difference of the data within the window in the time domain as a data rate of change feature; and simultaneously identifying and recording the location and value of local extrema exceeding a preset threshold within the window as the feature point sampling value, where N is a positive integer.

3. The method according to claim 1, characterized in that, The dynamic selection of compression combination algorithm in step S2 is specifically as follows: the multi-dimensional dynamic feature vector extracted in step S1, together with the current available network bandwidth and the target compression ratio, is input into a pre-trained classification and regression decision tree model; the classification and regression decision tree model first outputs a compression mode classification, which corresponds to a compression algorithm sequence framework, and then outputs a set of specific parameter configurations for each algorithm step in the compression algorithm sequence framework, and dynamically selects the compression combination algorithm according to the specific parameter configurations.

4. The method according to claim 3, characterized in that, Step S2 also includes online fine-tuning of the decision tree model. During the compression process, the actual compression ratio and compression speed are collected in real time. When the deviation between the actual compression ratio and the target compression ratio continues to exceed the set range, or the compression speed is lower than expected, the decision threshold or output value of the corresponding node of the current decision tree model is corrected online.

5. The method according to claim 2, characterized in that, Step S3, which involves processing the high-frequency test data using a compression and combination algorithm, specifically includes: dividing the high-frequency test data into overlapping frames and non-overlapping frames according to the data continuity requirements of the selected compression and combination algorithm; and labeling each frame of data with a type label based on the spectral characteristics, complexity characteristics, and data change rate characteristics, wherein the type label includes stationary frames, transient frames, and periodic frames.

6. The method according to claim 5, characterized in that, Step S3 also includes a differential preprocessing step corresponding to the frame type label, specifically including: for data labeled as stationary frames, a moving average filter is used for smoothing to reduce random noise; for data labeled as transient frames, an edge-preserving filter is used for noise reduction while maintaining the edges of abrupt changes, and the start and end points of abrupt changes are marked with high precision; for data labeled as periodic frames, a base period estimation and phase alignment operation based on the autocorrelation function is performed to improve the efficiency of the periodic compression algorithm.

7. The method according to claim 1, characterized in that, The intermediate correction step in step S4 specifically includes: performing statistical characteristic analysis on the output data block of the previous algorithm, calculating its data distribution histogram, mean, and variance; comparing the statistical characteristics with the ideal input data distribution range preset for the next algorithm in the currently selected compression combination algorithm; if there is a deviation, transforming the data block through a numerical mapping function to adjust its statistical characteristics to the preset ideal range, and then inputting the adjusted data block into the next algorithm.

8. The method according to claim 1, characterized in that, The transmission state matrix mentioned in step S5 is a three-dimensional matrix, whose three dimensions represent different transmission priority levels of data packets, different alternative network link identifiers, and a link quality score calculated by combining real-time network latency, packet loss rate, and estimated throughput. Based on the transmission priority of the data packets, the corresponding two-dimensional submatrix is ​​selected in the matrix. Then, a preset algorithm is used to select the link with the best link quality score and not currently at full load for each data packet in the two-dimensional submatrix for binding.

9. The method according to claim 8, characterized in that, The process of adding redundant forward error correction codes in step S5 is dynamically related to link allocation. First, the system maintains a lookup table of link packet loss rate and required error correction redundancy. After allocating a specific link to a data packet, the system immediately queries the real-time packet loss rate of that link and dynamically determines the type and redundancy of the error correction code to be added based on the lookup table. For high-priority data packets allocated to links with high packet loss rates, a coding method with strong error correction capability but high overhead is adopted. For low-priority data packets allocated to high-quality links, lightweight check or even no error correction code is added.

10. The method according to claim 1, characterized in that, The receiving end first uses the redundant forward error correction code to detect and correct errors in the received data packets. Correctable errors are directly repaired, while errors exceeding the error correction capability are marked as suspicious packets. Based on the corrected data and compression strategy information, preliminary decompression is performed to obtain the reconstructed data stream to be verified. According to the feature point location index information associated with the sending end and the compressed data stream, each feature point is precisely located in the reconstructed data stream to be verified. The actual sampled values ​​at the corresponding positions in the reconstructed data stream are extracted to form a sequence of reconstructed feature point sampled values. The sequence of reconstructed feature point sampled values ​​is compared with the original feature point sampled value sequence synchronized by the sending end. The comparison dimensions include absolute numerical error, local waveform similarity, and statistical distribution consistency. Only when the comparison results of all dimensions meet their respective preset tolerance thresholds is the transmission and reconstruction process of the batch of data determined to be lossless. If any dimension comparison fails, a retransmission request for the suspicious data packet or corresponding data segment is triggered.

11. A system for implementing the adaptive compression and lossless fast transmission method for high-frequency test data according to any one of claims 1-10, characterized in that, include: The dynamic feature extraction module is used to perform real-time time-frequency domain analysis on the input high-frequency test data, extract multi-dimensional dynamic feature vectors, and mark the feature point sampling values; The compression decision module is used to dynamically select a compression combination algorithm based on the extracted multidimensional dynamic feature vector, combined with the currently available network bandwidth and the target compression ratio. A data preprocessing module is used to process the high-frequency test data according to the compression and combination algorithm; A multi-level compression execution module is used to execute the compression combination algorithm, and use the output of the previous level algorithm as the input of the next level algorithm. An intermediate correction step is inserted between adjacent algorithms to adjust the input parameters of the next level algorithm to obtain the compressed data packet. The transmission scheduling and protection module is used to construct a transmission state matrix based on transmission priority, network latency and packet loss rate, dynamically allocate the compressed data packets to different network links based on the transmission state matrix, and add redundant forward error correction codes. The lossless verification and reconstruction module is used to reverse the decompression after the compressed data packet is received on the different network links, and to realize the lossless verification of the high-frequency test data by comparing the feature point sampling value with the redundant forward error correction code.

12. An electronic device having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the adaptive compression and lossless fast transmission method for high-frequency test data as described in any one of claims 1 to 10.

Images