Water conservancy monitoring data compression transmission method and system based on edge cloud cooperation

Abstract

The application discloses a water conservancy monitoring data compression transmission method based on edge cloud cooperation and belongs to the technical field of network data transmission and edge cloud cooperation calculation, which comprises the following steps: acquiring original data output by a water conservancy monitoring source and generating a monitoring data frame, generating a segment importance vector through segment division and feature extraction, generating a compression control vector by fusing link states and edge resource parameters, performing hierarchical expression according to the importance, executing structured compression and hierarchical packaging, performing hierarchical sending according to the link states after generating a verifiable packaging bag, generating strategy feedback information after the cloud completes integrity verification and decompression recovery, and dynamically updating compression parameters by the edge according to the feedback. The compression control vector is generated by adopting segment importance evaluation and transmission constraint fusion, hierarchical expression and differential compression are performed on data, and reliable priority transmission of key information is realized through hierarchical verification and hierarchical sending strategies, so that the efficiency and reliability of water conservancy monitoring data transmission can be ensured in a dynamic network environment.

Description

Technical Field

[0001] This invention relates to the field of network data transmission and edge-cloud collaborative computing technology, and in particular to a method and system for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration. Background Technology

[0002] A water conservancy monitoring system is a distributed system that collects physical parameters such as water level, flow velocity, and rainfall through sensors deployed at water conservancy project sites, and uploads the data to a cloud platform for analysis and decision-making. With the development of the Internet of Things and edge computing technologies, the edge-cloud collaborative architecture has become the mainstream deployment model in the field of water conservancy monitoring, with edge devices responsible for data collection and preliminary processing, and the cloud responsible for large-scale storage and in-depth analysis.

[0003] In existing technologies, the transmission of water conservancy monitoring data often employs a fixed compression ratio or a uniform compression algorithm to process all data. These approaches typically apply the same compression intensity to the entire data stream, failing to differentiate the importance of data content or consider dynamic changes in network conditions and equipment load. Some solutions perform data compression in the cloud before transmitting it back to the edge, increasing latency and failing to effectively utilize edge computing resources.

[0004] The existing technical solutions have the following technical defects: First, indiscriminate compression of data from heterogeneous data sources leads to distortion of key information at high compression ratios, affecting the accuracy of subsequent analysis; second, they fail to dynamically adjust compression and transmission strategies according to real-time network conditions and device resource status, and cannot guarantee priority transmission of key data when the network is congested or the device is under high load; third, they lack a hierarchical verification mechanism for data importance, and the entire data frame needs to be retransmitted when a transmission error occurs, resulting in bandwidth waste. Summary of the Invention

[0005] To address the aforementioned issues, this invention provides a method and system for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration. It employs a fusion of fragment importance assessment and transmission constraints to generate a compression control vector, performs hierarchical representation and differentiated compression on the data, and achieves reliable priority transmission of key information through hierarchical verification and graded transmission strategies. This ensures the efficiency and reliability of water conservancy monitoring data transmission in dynamic network environments.

[0006] The above objectives can be achieved through the following approach: A method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration includes: acquiring raw monitoring data output from a water conservancy monitoring source; performing time alignment and data cleaning to obtain monitoring data frames and calculating frame-level summary information of the monitoring data frames to generate frame fingerprints; parsing the monitoring data frames and dividing them into segments according to data type and change characteristics to obtain a segment set; extracting features from the segment set to generate a segment importance vector; acquiring link state parameters and edge resource state parameters for fusion representation to generate a transmission constraint vector; fusing the transmission constraint vector with the segment importance vector to generate a compression control vector; and then, based on the segment... The importance vector is used to hierarchically divide the fragment set to generate a hierarchical fragment set. Hierarchical expression data is constructed based on the compression control vector and the hierarchical fragment set. Structured compression is performed on the hierarchical expression data, and hierarchical encapsulation is performed in combination with the frame fingerprint to generate hierarchical encapsulation units. Verifiable encapsulation packets are generated based on the hierarchical encapsulation units. A hierarchical transmission strategy is executed on the verifiable encapsulation packets based on the link state parameters to generate transmission logs and send them to the cloud. The verification results and strategy feedback information returned by the cloud are received and parsed to generate a strategy update quantity and update the generation rules of the compression control vector based on the strategy update quantity.

[0007] Optionally, the step of acquiring the raw monitoring data output from the water conservancy monitoring source, performing time alignment and data cleaning to obtain monitoring data frames, and calculating the frame-level summary information of the monitoring data frames to generate frame fingerprints includes: acquiring the sampling time identifier and data fields of each water conservancy monitoring source to generate raw monitoring data; performing time alignment on the raw monitoring data and performing consistency completion on missing fields to generate aligned data sequences; performing noise suppression and outlier screening on the aligned data sequences to construct monitoring data frames; and extracting frame-level summary information from the monitoring data frames and performing summary encoding to generate frame fingerprints.

[0008] Optionally, the step of parsing the monitoring data frame and dividing it into segments according to data type and change characteristics to obtain a segment set, and extracting features from the segment set to generate a segment importance vector includes: obtaining the field structure and field semantic identifier of the monitoring data frame, and generating field structure information; dividing the monitoring data frame into segments according to the field structure information and recording the segment boundaries to generate a segment set; extracting change intensity features and stability features from the segment set to generate a change feature set; and evaluating the importance of the change feature set to obtain a segment importance vector.

[0009] Optionally, the step of fusing the obtained link state parameters and edge resource state parameters to generate a transmission constraint vector, and then fusing the transmission constraint vector with the segment importance vector to generate a compression control vector, includes: obtaining link state parameters and performing time-series aggregation to generate a link state feature vector; obtaining edge resource state parameters and performing load characterization to generate a resource state feature vector; fusing the link state feature vector and the resource state feature vector to generate a transmission constraint vector; and jointly mapping the transmission constraint vector with the segment importance vector to generate a compression control vector.

[0010] Optionally, the step of hierarchically dividing the fragment set according to the fragment importance vector to generate a hierarchical fragment set, and constructing hierarchical expression data based on the compression control vector and the hierarchical fragment set includes: hierarchically dividing the fragment set according to the fragment importance vector to generate a hierarchical fragment set containing key layer fragments and non-key layer fragments; performing fidelity expression construction on the key layer fragments in the hierarchical fragment set to generate key layer expression data; performing approximate expression construction on the non-key layer fragments in the hierarchical fragment set while retaining trend description information to generate non-key layer expression data; and merging the key layer expression data and the non-key layer expression data to generate hierarchical expression data.

[0011] Optionally, the step of performing structured compression on the hierarchical representation data and performing hierarchical encapsulation in conjunction with the frame fingerprint to generate a hierarchical encapsulation unit includes: obtaining hierarchical description information of the hierarchical representation data and selecting a compression path to generate hierarchical compression parameters; performing structured compression on the hierarchical representation data according to the hierarchical compression parameters to generate a compression payload; and writing the compression payload into the hierarchical description information and associating it with the frame fingerprint to generate a hierarchical encapsulation unit.

[0012] Optionally, the step of generating a verifiable encapsulated packet based on the layered encapsulation unit, executing a hierarchical transmission strategy on the verifiable encapsulated packet based on the link state parameters, generating a transmission log, and sending it to the cloud includes: calculating a layered checksum based on the layered encapsulation unit to generate a checksum encapsulation unit; performing packet segmentation and sequence marking on the checksum encapsulation unit to generate a verifiable encapsulated packet; performing transmission queue shaping on the verifiable encapsulated packet based on the link state parameters to generate a transmission queue; executing a hierarchical transmission strategy based on the transmission queue and recording the transmission results, generating a transmission log, and sending it to the cloud.

[0013] Optionally, the method further includes a cloud processing step, which includes: receiving the verifiable encapsulated packet and performing integrity verification according to the verification encapsulation unit to generate a verification result; when the verification result indicates that the verification passed, decapsulating the verifiable encapsulated packet to generate a layered payload and performing decompression and recovery on the layered payload to generate a recovery monitoring data frame; when the verification result indicates that the verification failed, performing error location on the verifiable encapsulated packet to generate error location information and forming a retransmission request; performing availability assessment on the recovery monitoring data frame and generating the policy feedback information in combination with link statistics results, and sending it to the edge.

[0014] Optionally, the step of receiving and parsing the verification result and policy feedback information returned by the cloud, generating a policy update amount, and updating the generation rule of the compression control vector based on the policy update amount includes: receiving the verification result and policy feedback information returned by the cloud, performing consistency analysis, and generating a feedback feature vector; performing correlation analysis between the feedback feature vector and the sending logs to generate a bottleneck attribution result; calculating the policy update amount based on the bottleneck attribution result, updating the generation rule of the compression control vector, and outputting update confirmation information.

[0015] Based on the same inventive concept, this invention also provides a method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration, wherein the system includes an edge terminal and a cloud terminal; wherein, The edge includes: a data framing module, used to acquire raw monitoring data output from the water conservancy monitoring source, perform time alignment and data cleaning to obtain monitoring data frames, calculate frame-level summary information of the monitoring data frames, and generate frame fingerprints; an importance assessment module, used to parse the monitoring data frames and divide them into segments according to data type and change characteristics to obtain a segment set and extract features from the segment set to generate a segment importance vector; a constraint fusion module, used to acquire link state parameters and edge resource state parameters, fuse them to generate a transmission constraint vector, and fuse the transmission constraint vector with the segment importance vector to generate a compression control vector; and a hierarchical expression module, used to classify the segment set according to the segment importance vector. The system performs hierarchical partitioning to generate a hierarchical fragment set, and constructs hierarchical representation data based on the compression control vector and the hierarchical fragment set. A compression and encapsulation module performs structured compression on the hierarchical representation data and performs hierarchical encapsulation based on the frame fingerprint to generate hierarchical encapsulation units. A verification and sending module generates verification and encapsulation units based on the hierarchical encapsulation units, generates verifiable encapsulation packets based on the verification and encapsulation units, executes a hierarchical sending strategy on the verifiable encapsulation packets based on the link state parameters, generates sending logs, and sends them to the cloud. A policy update module receives and parses the verification results and policy feedback information returned from the cloud, generates a policy update quantity, and updates the generation rules of the compression control vector based on the policy update quantity. The cloud includes: a verification and decapsulation module, used to receive the verifiable encapsulated packet and perform integrity verification according to the verification and encapsulation unit, generating a verification result; when the verification result indicates that the verification is passed, decapsulating the verifiable encapsulated packet, generating a layered payload, and performing decompression and recovery on the layered payload to generate a recovery monitoring data frame; an error location module, used to locate errors in the verifiable encapsulated packet when the verification result indicates that the verification is failed, generating error location information and forming a retransmission request; and a feedback generation module, used to perform availability assessment on the recovery monitoring data frame and generate the policy feedback information in combination with link statistics results, and send it to the edge.

[0016] Compared with the prior art, the present invention has the following advantages: This invention uses a hierarchical representation and differentiated compression mechanism to dynamically adjust the compression strategy based on data importance and real-time transmission constraints. This reduces network bandwidth usage while ensuring the integrity of key information and improves data transmission efficiency in edge-cloud collaborative scenarios.

[0017] This invention constructs a closed-loop feedback mechanism and uses availability assessment and link statistics results issued by the cloud to adaptively update the edge compression parameters, enabling the system to continuously adapt to network fluctuations and changes in data characteristics, thereby enhancing the robustness and intelligence of the transmission strategy.

[0018] This invention employs layered verification and error location technology to achieve independent integrity verification of critical layer data and non-critical layer data. When a transmission error occurs, it can accurately locate the damaged data and perform differentiated retransmission, reducing redundant retransmission overhead and improving transmission reliability.

[0019] This invention deeply integrates data importance assessment, network state awareness, and edge resource load characterization to generate a compression control vector that guides a tiered transmission strategy. This ensures that critical information is transmitted first when the network is congested, effectively addressing the challenges of resource constraints and link instability in edge-cloud collaborative scenarios. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the water conservancy monitoring data compression and transmission method based on edge-cloud collaboration according to an embodiment of the present invention.

[0021] Figure 2 This is a comparison chart of the noise suppression effects of aligned data sequences in embodiments of the present invention.

[0022] Figure 3 This is a time-series convergence curve of the link state parameters in an embodiment of the present invention.

[0023] Figure 4 This is a schematic diagram of the closed-loop adaptive update process of the compression control vector generation rule in an embodiment of the present invention.

[0024] Figure 5 This is a schematic diagram of the structure of the water conservancy monitoring data compression and transmission system based on edge-cloud collaboration according to an embodiment of the present invention.

[0025] Figure 6 This is a schematic diagram of the framework of the water conservancy monitoring data compression and transmission system based on edge-cloud collaboration according to an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Reference Figure 1One embodiment of the present invention proposes a water conservancy monitoring data compression and transmission method based on edge-cloud collaboration. It adopts fragment importance assessment and transmission constraint fusion to generate compression control vector, performs hierarchical expression and differentiated compression on data, and achieves reliable priority transmission of key information through hierarchical verification and graded transmission strategy. It can ensure the efficiency and reliability of water conservancy monitoring data transmission in dynamic network environment.

[0028] The method described in this embodiment specifically includes: S1. Obtain the raw monitoring data output from the water conservancy monitoring source, perform time alignment and data cleaning to obtain monitoring data frames, calculate the frame-level summary information of the monitoring data frames, and generate frame fingerprints; In one embodiment of the present invention, step S1 includes the following steps: Obtain the sampling time identifier and data fields of each water conservancy monitoring source to generate raw monitoring data; The original monitoring data is time-aligned and missing fields are filled with consistency to generate an aligned data sequence. Noise suppression and outlier screening are performed on the aligned data sequence to construct a monitoring data frame; Frame-level summary information is extracted from the monitoring data frame and the summary is encoded to generate a frame fingerprint.

[0029] Specifically, the goal of this step is to process heterogeneous and asynchronous raw data from different water monitoring sources, transforming them into well-organized, clean data units with unique identifiers—monitoring data frames with frame fingerprints—laying the foundation for subsequent intelligent compression and transmission. Water monitoring sources are various sensors or measuring devices deployed at water conservancy project sites, such as water level gauges, flow meters, and rain gauges. Each water monitoring source generates a series of discrete measurement readings during operation. Each reading contains two core parts: a sampling time identifier and a data field. The sampling time identifier records the precise moment of data acquisition, while the data field carries the specific physical quantity measurement value. Combining these readings, containing sampling time identifiers and data fields, from all water monitoring sources constitutes the raw monitoring data.

[0030] Due to differences in sampling frequencies and start-up times among various water conservancy monitoring sources, the raw monitoring data is not aligned in the time dimension, and data fields at certain time points may be missing due to equipment failure or communication interruptions. To address this issue, it is necessary to perform time alignment on the raw monitoring data and perform consistency completion on missing fields. Time alignment unifies all data onto a standardized time grid, such as one time point per second. For fields where no data was collected at the standard time point, consistency completion techniques are used for interpolation to fill in the missing data, thereby generating a temporally continuous and complete aligned data sequence. A linear interpolation method can be used for consistency completion, and its calculation method is as follows: , In this formula, Represents the standard time point Data field values ​​that need to be completed. and They are distances The most recent previous and next actual sampling time identifiers. and These are the data field values ​​corresponding to the two actual sampling time markers. The formula constructs a straight line using two known points and finds the target time point along this line. The corresponding values ​​ensure a smooth transition for completing the data.

[0031] like Figure 2 As shown, after obtaining the aligned data sequence, the data may still contain noise introduced by the sensor itself or environmental interference, as well as outliers caused by sudden events or equipment malfunctions. Therefore, noise suppression and outlier screening need to be performed on the aligned data sequence. Noise suppression aims to smooth the data sequence and filter out high-frequency fluctuations; a common method is moving average filtering. Outlier screening is used to identify and process data points that far exceed the normal fluctuation range. After these two processing steps, the data quality is improved, forming a data segment that is temporally regular and numerically reliable; this data segment is then constructed as a monitoring data frame. The calculation process of moving average filtering can be expressed as: , in, It is the first in the aligned data sequence Data points The new value obtained after noise suppression. The radius of the sliding window determines the degree of smoothness. Its value is set based on the analysis of 200 sets of measured data from industrial sensors. Setting it to 2 effectively filters out common Gaussian white noise while preserving the basic trend of the data. This formula calculates the smoothness of the data at a point... Centered The arithmetic mean of all data points within the length window is used to replace the original value, thus achieving a smoothing effect.

[0032] To generate a compact and unique identifier, or frame fingerprint, for each monitoring data frame, the process first extracts frame-level summary information. This information consists of numerical values ​​that summarize the core statistical characteristics of the entire data frame, such as mean, standard deviation, maximum, and minimum values. Then, this frame-level summary information is digested and encoded into a fixed-length string. This string is the frame fingerprint, which not only uniquely identifies a monitoring data frame but also serves for subsequent data verification and rapid retrieval. The process of extracting frame-level summary information and generating frame fingerprints can be represented as follows: , In this expression, This represents the final generated frame fingerprint. This represents a hash function, such as the secure hash algorithm SHA-256, which maps inputs of arbitrary length to outputs of fixed length. Input vector This refers to frame-level summary information, in which It is the arithmetic mean of all data points within the monitored data frame. Standard deviation is used to measure the dispersion of data. and These are the maximum and minimum values ​​within the data frame, respectively. Through hash operations, even slight changes in the frame-level digest information will result in significant differences in the generated frame fingerprint, ensuring its uniqueness and tamper sensitivity.

[0033] For example, suppose there are two water monitoring sources: a water level gauge (source A) and a flow velocity meter (source B). During a certain time period, the system acquires the following raw monitoring data: Source A (water level): at timestamp 1.0, the data field is 5.20 meters; at timestamp 3.0, the data field is 5.40 meters. Source B (flow velocity): at timestamp 1.5, the data field is 10.1 cubic meters per second; at timestamp 3.5, the data field is 10.7 cubic meters per second.

[0034] Next, time alignment and consistency completion were performed on the original monitoring data, setting the standard time grid to 1.0 second, 2.0 seconds, and 3.0 seconds. For source A, the data field at timestamp 2.0 seconds was missing, and consistency completion was performed using a linear interpolation formula: For source B, the data fields at timestamps 2.0 and 3.0 are missing and are also filled in. After processing, an aligned data sequence is generated, for example, the water level of source A at timestamps 1.0, 2.0, and 3.0 are 5.20 meters, 5.30 meters, and 5.40 meters, respectively.

[0035] Suppose that the actual reading of source A at timestamp 2.0 is disturbed and becomes 5.90 meters, resulting in a sequence [5.20, 5.90, 5.40]. A moving average filter is used (with a set window radius). Noise suppression is applied to the second point: The processed sequence becomes [5.20, 5.50, 5.40], which is then used to construct a monitoring data frame.

[0036] Finally, frame-level summary information is extracted from this monitoring data frame, and a frame fingerprint is generated. The frame-level summary information is then calculated as follows: mean. Meters. Standard deviation Meters. Maximum value. Meters. Minimum value The digest information is combined into a vector [5.37, 0.12, 5.50, 5.20], and then a unique frame fingerprint, such as "c3a2…e9f1", is generated through digest encoding, for example, by applying the SHA-256 hash function. This frame fingerprint is then firmly bound to the processed monitoring data frame.

[0037] S2. Parse the monitoring data frame and divide it into segments according to data type and change characteristics to obtain a segment set. Extract features from the segment set to generate a segment importance vector. In one embodiment of the present invention, step S2 includes the following steps: Obtain the field structure and field semantic identifiers of the monitoring data frame, and generate field structure information; The monitoring data frame is segmented based on the field structure information, and the segment boundaries are recorded to generate a segment set; Extract change intensity features and stability features from the fragment set to generate a change feature set; The importance of the set of changing features is evaluated to obtain the fragment importance vector.

[0038] Specifically, after generating standardized monitoring data frames through step S1, this step aims to deeply analyze the internal structure and data dynamics of the monitoring data frames. By quantitatively assessing the importance of different data components, it provides a decision-making basis for subsequent differentiated compression and transmission. First, it is necessary to obtain the field structure and semantic identifiers of the monitoring data frames to generate field structure information. Monitoring data frames are formed by fusing data from different water conservancy monitoring sources. The field structure information is like a manual for this data, detailing the source, physical meaning, and data type of each column of data within the data frame. For example, it would specify that the first column is the water level gauge reading, in meters, with a floating-point data type; the second column is the current meter reading, in cubic meters per second, also with a floating-point data type. This field structure information is predefined and provides the foundation for subsequent data parsing and segmentation.

[0039] Based on the generated field structure information, the monitoring data frame is then segmented and segment boundaries are recorded, thereby generating a segment set. Segmentation involves splitting a complete but mixed monitoring data frame according to data type or source, forming multiple homogeneous data sequences. Each such sequence is called a segment. For example, a monitoring data frame containing both water level and flow velocity data will be divided into a water level segment and a flow velocity segment. During the segmentation process, the start and end positions of each segment in the original data frame, i.e., the segment boundaries, are precisely recorded. All these segments together constitute the segment set.

[0040] Having obtained the set of data segments, the next step is to extract quantitative indicators of the dynamic characteristics of each segment, namely, to extract the intensity of change and stability features, and jointly generate a set of change features. The intensity of change features measure the overall fluctuation amplitude of the data within a segment, while the stability features reflect the smoothness of the data's changes. Intensity of Change Features The calculation method is as follows: , In this formula, It is the index of the segment. and These are fragments The maximum and minimum values ​​of the data points within the range, and the difference between them, represent the dynamic range of the data. It is a fragment The arithmetic mean of all data points within the range is used to normalize the dynamic range, thereby eliminating the influence of different physical dimensions and making the data characteristics of different types of sensors comparable. Stability characteristics. The calculation method is as follows: , in, It is a fragment The standard deviation of the data points within a range measures how much the data points revolve around the mean. The degree of dispersion. This ratio is called the coefficient of variation, and it is dimensionless. Subtracting this value from 1 makes... A higher value indicates more stable data with less fluctuation. The calculated value for each segment... and When combined, they form a set of changing characteristics.

[0041] Finally, the importance of the set of changing features needs to be evaluated to obtain a segment importance vector. This evaluation process aims to fuse multiple features of each segment into a single importance score. Generally, data segments that change drastically or are less stable are considered to contain more information and are therefore more important. Importance Score The calculation formula is as follows: , In this formula, It is a fragment The final importance score. and These are preset weighting coefficients, representing the intensity of change and the importance of instability, respectively, and satisfying the following conditions: The setting of these two weights is based on business needs. For example, in a flood warning scenario, drastic changes in water level are more noteworthy than steady fluctuations, therefore, weights can be set accordingly. It is 0.7. The value is 0.3, a setting derived from the analysis of 200 sets of measured data from industrial sensors during historical flood events. The formula... The term is equivalent to This directly reflects the instability of the data. All fragments... The values ​​are arranged in order to form the final output, which is the fragment importance vector.

[0042] For example, following the example from step S1, suppose a monitoring data frame contains data from 10 time points, with the data coming from a water level gauge (source A) and a flow meter (source B). First, obtain the field structure information, which indicates that the monitoring data frame contains two fields: field 1 is "water level", in "meters"; field 2 is "flow velocity", in "cubic meters per second".

[0043] Based on this field structure information, the monitoring data frame is segmented to generate a segment set. Segment 1 (water level): [5.20, 5.50, 5.40, 5.42, 5.41, 5.39, 6.50, 6.55, 6.52, 6.51]; Segment 2 (flow velocity): [10.1, 10.2, 10.1, 10.3, 10.2, 10.1, 12.5, 12.6, 12.4, 12.5]. This set containing the two segments is the segment set.

[0044] Next, the intensity and stability features of change are extracted from the fragment set to generate a set of change features. For fragment 1 (water level): maximum value. minimum value ;average value Standard deviation ; Characteristics of change intensity Stability characteristics .

[0045] For segment 2 (flow rate): maximum value minimum value ;average value Standard deviation ; Characteristics of change intensity Stability characteristics Therefore, the generated set of change features is {( ),( )}.

[0046] Finally, the importance of the set of changing features is evaluated to obtain a segment importance vector. Weights are then assigned. , The Importance of Segment 1 The importance of segment 2 The final generated segment importance vector is [0.1905, 0.1861]. This vector quantifies that, in this monitoring data frame, the water level segment is slightly more important than the flow velocity segment.

[0047] S3. Obtain the link status parameters and the edge resource status parameters, fuse them to generate a transmission constraint vector, and fuse the transmission constraint vector with the segment importance vector to generate a compression control vector; In one embodiment of the present invention, step S3 includes the following steps: Obtain link status parameters and perform time-series aggregation to generate link status feature vectors; Obtain edge resource status parameters and perform load characterization to generate resource status feature vectors; The link state feature vector and the resource state feature vector are fused to generate a transmission constraint vector; The transmission constraint vector and the fragment importance vector are jointly mapped to generate a compression control vector.

[0048] Specifically, such as Figure 3 As shown, after quantifying the importance of the data content itself in step S2, this step introduces constraints from the external environment and device capabilities, aiming to generate a comprehensive control signal that can dynamically guide subsequent compression strategies. The core of this process is to combine the importance of the data with the realities of transmission and processing to achieve intelligent decision-making. This process begins with acquiring link state parameters and performing time-series aggregation to generate a link state feature vector. Link state parameters are instantaneous indicators describing the quality of the network communication channel from the edge device to the cloud server, mainly including bandwidth, latency, and packet loss rate. Because these parameters fluctuate in real time, directly using instantaneous values ​​may lead to unstable decisions. Therefore, time-series aggregation is needed to obtain a smoother feature that better represents recent network trends by analyzing historical data over a period of time. An exponentially weighted moving average method can be used for this purpose, calculated as follows: , In this formula, At the current moment Link state characteristics after aggregation, such as the aggregated bandwidth. It is the current moment. The instantaneous link status parameter values ​​collected. It is the result of the convergence of the previous moment. It is a smoothing coefficient, with a value between 0 and 1, which determines the weight of newly collected parameter values ​​in the aggregation results. The setting is based on the analysis of 200 sets of measured data under different network environments. Setting it to 0.3 can effectively suppress instantaneous jitter while ensuring sensitivity. This calculation is performed on bandwidth, latency, and packet loss rate separately to obtain the aggregated bandwidth, latency, and packet loss rate, which together constitute the link state feature vector.

[0049] Simultaneously, the system acquires edge resource status parameters and performs load characterization, generating a resource status feature vector. Edge resource status parameters reflect the current workload of the edge computing device performing data processing and compression tasks, typically including CPU utilization and memory utilization. Load characterization aims to transform these raw percentage readings into standardized metrics that reflect the device's processing capacity. This process is typically a normalization process. , in, It is the normalized resource utilization rate, such as CPU utilization rate. This is the currently measured resource usage, while This represents the total amount of the resource. Combining the normalized CPU utilization and memory utilization together forms the resource state feature vector.

[0050] Next, the link state feature vector and resource state feature vector need to be fused to generate a transmission constraint vector. This step integrates information from two dimensions—network condition and device condition—into a single constraint metric that measures overall transmission and processing capabilities. The fusion process can be a weighted summation, but it requires first making the parameters with different physical meanings dimensionless and unifying their trends. For example, higher bandwidth is better, while lower latency, packet loss rate, and resource utilization are better. Transmission Constraint Metric The calculation formula is as follows: , In this formula, This represents the final transmission constraint index; the higher the value, the better the overall conditions. , , These are the normalized bandwidth, latency, and packet loss rate, respectively. It is a comprehensive resource load index, which can be obtained by weighted summation of the elements in the resource status feature vector. These are their respective weighting coefficients, which sum to 1. These weights are set based on the sensitivity of the specific application scenario to different constraints. For example, for tasks with high real-time requirements, latency has a higher weight. It will be set higher. This single metric. This constitutes the transmission constraint vector.

[0051] Finally, the transmission constraint vector and the segment importance vector obtained in step S2 are jointly mapped to generate a compression control vector. The purpose of this mapping is to generate a specific compression control parameter for each data segment, which directly guides the subsequent compression intensity. The logic of the joint mapping is: the more important the data segment, or the better the transmission constraints, the lower the compression intensity should be; conversely, the less important the data segment, or the worse the transmission constraints, the higher the compression intensity should be. Compression Control Parameter The calculation can be expressed as: , In this formula, This is a fragment The higher the value of the generated compression control parameter, the higher the compression intensity. It is a fragment Importance score. It is a transmission constraint indicator. It is the theoretical minimum value of the transmission constraint index. Make sure the denominator is positive. It is a global scaling factor used to... Adjust the parameters to the range acceptable to a specific compression algorithm, based on the performance curve of the compression algorithm. It is a very small positive number, for example This is used to prevent the denominator from being zero. All segments The values ​​are arranged in order to form the final compression control vector.

[0052] For example, following the example from step S2, the segment importance vector is known to be [0.1905, 0.1861]. Now, obtain the link state parameters. Assume the bandwidth collected at the three most recent time points is [500, 510, 495] kbps, and the latency is [120, 130, 115] ms. Use the exponentially weighted moving average (... (Perform time-series aggregation. Assuming the aggregation bandwidth at the previous moment was 505kbps, the current aggregation bandwidth is...) kbps. Similarly, the aggregation latency was calculated to be 120ms, and the aggregation packet loss rate was 1%. The generated link state feature vector is [502, 120, 0.01]. At the same time, the edge resource state parameters were obtained, with CPU utilization at 65% and memory utilization at 40%. After load characterization and normalization, the resource state feature vector is [0.65, 0.40].

[0053] The link state feature vector and resource state feature vector are merged and first normalized, assuming a bandwidth range of [0, 1000] kbps, a latency range of [0, 500] ms, a packet loss rate range of [0, 1], and a resource utilization rate range of [0, 1]. After normalization, , , If we set the resource weights to 0.7 for CPU and 0.3 for memory, then the overall resource load will be... Then set the transmission constraint weights. Calculate the transmission constraint index: The generated transmission constraint vector is [0.0693].

[0054] Finally, the transmission constraint vector and the fragment importance vector are jointly mapped, and a scaling factor is set. Theoretical minimum constraint (When all negative indicators are at their maximum and positive indicators are at their minimum) Calculate the compression control parameters for segment 1 (water level): Calculate the compression control parameters for segment 2 (flow rate): The final compression control vector is [49.09, 50.25]. This vector will guide subsequent steps to apply different compression intensities to the water level and flow velocity segments.

[0055] S4. Divide the fragment set into hierarchical levels according to the fragment importance vector to generate a hierarchical fragment set, and construct hierarchical expression data based on the compression control vector and the hierarchical fragment set; In one embodiment of the present invention, step S4 includes the following steps: The fragment set is hierarchically divided according to the fragment importance vector to generate a hierarchical fragment set containing key layer fragments and non-key layer fragments; Fidelity representation construction is performed on the key layer fragments in the hierarchical fragment set to generate key layer representation data; Approximate representation construction is performed on the non-critical layer fragments in the hierarchical fragment set while preserving trend description information, generating non-critical layer representation data; The key layer expression data and the non-key layer expression data are merged to generate hierarchical expression data.

[0056] Specifically, this step involves assessing the importance of the data content and quantifying environmental constraints. Its core objective is to preprocess the raw data based on this high-level decision-making information, reconstructing it into a hierarchical structure to prepare for subsequent differentiated compression and transmission. First, the fragment set is hierarchically divided according to the fragment importance vector generated in step S2. This division is achieved by setting an importance threshold, dividing all data fragments into two categories, generating a hierarchical fragment set containing critical and non-critical fragments. Fragments with importance above the threshold are classified as critical fragments; these fragments typically contain critical mutation information or core business data, requiring high-fidelity transmission. Fragments with importance below the threshold are classified as non-critical fragments; these fragments exhibit gradual or minor data changes and can tolerate a certain degree of information loss. The division rules are as follows: When... Time, fragment Belongs to key layer fragments; when At that time, fragment This belongs to a non-critical layer segment. It is a fragment Importance score. It is an importance threshold, which can be set based on historical data analysis. For example, it can be set as the average importance score of all segments. This setting is based on statistical analysis of 200 sets of industrial sensor measured data and can dynamically adapt to the overall importance distribution of different data frames.

[0057] After generating the hierarchical fragment set, the system applies different representation construction strategies to fragments at different levels. For key-level fragments in the hierarchical fragment set, fidelity representation construction is performed to generate key-level representation data. Fidelity representation construction means that no information of the original data points is changed, preserving all data points within the key-level fragments intact. This ensures that the most important data information is not lost during transmission.

[0058] Conversely, for non-critical layer fragments in the hierarchical fragment set, approximate representation construction is performed while preserving trend description information to generate non-critical layer representation data. This process aims to represent the original data fragments with less data while retaining their macroscopic trends. The compressed control vector generated in step S3 is utilized here. Specifically, an error-tolerance-based curve simplification algorithm, such as the Douglas-Puk algorithm, is employed to reduce the number of data points. The algorithm's error tolerance... Directly from the corresponding parameters in the compression control vector Decide: , In this formula, It is applied to non-critical layer fragments. The approximate error tolerance has the same dimensions as the original data. It is obtained from the compression control vector and is specific to the fragment. Dimensionless compression control parameters. This is a dimensionless matching coefficient used to convert dimensionless control parameters into error tolerances with physical units. Its value is set based on the specific operational requirements for data accuracy. For example, for water level data in meters... It can be set to 0.002 meters. A larger value indicates poorer transmission conditions or less important data, thus leading to... The larger the value, the more simplified the data points become. After processing by the simplification algorithm, the remaining sparse data points constitute the trend description information, which is the non-critical layer representation data.

[0059] Finally, the key-layer and non-key-layer data are merged to generate hierarchical data. This merging process is not merely a simple data splicing; it involves constructing a structured data object. This object attaches metadata to each fragment, indicating its level (key or non-key) and its corresponding data (raw data points or trend descriptions). This structured hierarchical data clearly distinguishes information of varying importance, laying the foundation for subsequent structured compression.

[0060] For example, following the example from step S3, the known fragment set includes fragment 1 (water level) and fragment 2 (flow velocity), with fragment importance vectors of [0.1905, 0.1861] and compression control vectors of [49.09, 50.25]. The fragment set is then hierarchically divided based on the fragment importance vectors. An importance threshold is then calculated. The average of all importance scores: The comparison shows that fragment 1 is more important. Therefore, segment 1 (water level) is classified as a critical layer segment. The importance of segment 2... Therefore, segment 2 (flow velocity) is divided into a non-critical layer segment. The resulting hierarchical segment set contains one critical layer segment (water level) and one non-critical layer segment (flow velocity). A faithful representation is constructed for the critical layer segment (water level), with the original water level data being [5.20, 5.50, 5.40, 5.42, 5.41, 5.39, 6.50, 6.55, 6.52, 6.51]. The faithful representation directly retains all these data points, and the generated critical layer representation data is this complete sequence. An approximate representation is constructed for the non-critical layer segment (flow velocity). The original flow velocity data is [10.1, 10.2, 10.1, 10.3, 10.2, 10.1, 12.5, 12.6, 12.4, 12.5]. It is necessary to consider the compression control vector... To determine the degree of approximation, set the dimensional matching coefficients. cubic meters per second. Calculation error tolerance: cubic meters per second. Applying this error tolerance to the curve simplification algorithm, the algorithm will discard data points whose impact on the overall curve shape is less than 0.1005. Assume that after the algorithm executes, the retained trend description information consists of the start point of the sequence, an intermediate turning point, and the end point, for example, [10.1, 10.1, 12.6, 12.5]. These four points constitute the non-critical layer representation data.

[0061] Finally, the key layer representation data and the non-key layer representation data are merged to generate hierarchical representation data. This final result is a structured object with the following content: Segment 1 information: {Hierarchy: Key Layer, Data: [5.20, 5.50, 5.40, 5.42, 5.41, 5.39, 6.50, 6.55, 6.52, 6.51]}; Segment 2 information: {Hierarchy: Non-key Layer, Data: [10.1, 10.1, 12.6, 12.5]}. This hierarchical representation data completely retains the key information and effectively simplifies the secondary information.

[0062] S5. Perform structured compression on the hierarchical representation data, and combine it with the frame fingerprint for hierarchical encapsulation to generate a hierarchical encapsulation unit; In one embodiment of the present invention, step S5 includes the following steps: Obtain the hierarchical description information of the hierarchical representation data and select the compression path to generate hierarchical compression parameters; The hierarchical representation data is subjected to structured compression based on the hierarchical compression parameters to generate a compressed payload; The compressed payload is written into the hierarchical description information and associated with the frame fingerprint to generate a hierarchical encapsulation unit.

[0063] Specifically, after reconstructing the original monitoring data frame into structured hierarchical representation data in step S4, the goal of this step is to utilize this structure to perform actual, differentiated compression of the data and package it together with metadata and unique identifiers to form a compact, self-interpreting data unit, thus preparing for efficient transmission. The hierarchical description information of the hierarchical representation data is obtained, and a compression path is selected to generate hierarchical compression parameters. The hierarchical representation data embeds the hierarchical affiliation of each data segment, i.e., critical or non-critical layer; this is the hierarchical description information. Compression path selection assigns the most suitable compression algorithm to each segment based on this information. For data segments marked as critical layers, the system selects a lossless compression algorithm to ensure that the data can be completely and accurately recovered. For data segments marked as non-critical layers, a lossy compression algorithm is selected to achieve a higher compression ratio. The hierarchical compression parameters are the output of this selection process; they specify the compression algorithm to be used for each segment and the specific configuration required by that algorithm. For example, for lossy compression, the compression strength needs to be controlled by a quantization parameter, which is directly derived from the compression control vector generated in step S3. The quantization step size for lossy compression is also specified. With compression control parameters The relationship can be represented as: , In this formula, It is applied to non-critical layer fragments. The quantization step size has the same dimensions as the data within the segment. It is obtained from the compression control vector for the segment. Dimensionless control parameters. It is a scaling factor that maps dimensionless control parameters to the range of quantization steps with physical units required by the compression algorithm. The value is set based on the characteristics of the selected lossy compression algorithm and the business's tolerance for data accuracy. For example, based on a compression experiment using 200 sets of measured data from industrial sensors, for a compression path employing discrete cosine transform, the value is set as follows: A value of 0.001 can achieve a good balance between compression ratio and data distortion.

[0064] Based on the generated hierarchical compression parameters, the system then performs structured compression on the hierarchical representation data, generating a compression payload. Structured compression means that for each segment in the hierarchical representation data, its corresponding compression algorithm and parameters are applied independently. Data from critical layer segments are processed using lossless compression algorithms, such as the Lempel-Ziv-Welch (LZW) algorithm, which replaces recurring data sequences by building a dictionary. Data from non-critical layer segments are processed using lossy compression algorithms, such as Discrete Cosine Transform (DCT) combined with the quantization step size calculated using the above formula. Quantization is then performed. The binary data streams obtained after each segment undergoes its own compression process are aggregated together to form the compressed payload.

[0065] The final step is to write the compressed payload into the hierarchical description information and associate it with the frame fingerprint generated in step S1 to generate a hierarchical encapsulation unit. This process constructs a complete data packet that can be transmitted and parsed independently. The hierarchical encapsulation unit contains a header and a body. The header area stores metadata, including a frame fingerprint that uniquely identifies the entire data frame, and hierarchical description information to guide decompression, such as the hierarchical level of each segment, the compression algorithm used, and the quantization step size of non-critical layer segments. The body area stores the actual compressed payload. By organizing this information together, a well-structured and complete hierarchical encapsulation unit is formed. The receiving end only needs to parse this unit to complete the hierarchical recovery of the data.

[0066] For example, following the example from step S4, the currently processed hierarchical representation data is as follows: Fragment 1 information: {Level: Critical Layer, Data: [5.20, 5.50, 5.40, 5.42, 5.41, 5.39, 6.50, 6.55, 6.52, 6.51]}; Fragment 2 information: {Level: Non-Critical Layer, Data: [10.1, 10.1, 12.6, 12.5]}. Furthermore, the frame fingerprint is known to be “c3a2…e9f1”, and the compression control vector is [49.09, 50.25]. For fragment 1 (water level), its level is a critical layer, and the system selects a lossless compression path, such as the LZW algorithm. Its hierarchical compression parameters are {Algorithm: LZW}. For fragment 2 (flow velocity), its level is a non-critical layer, and the system selects a lossy compression path, such as the DCT algorithm. Its quantization step size needs to be calculated. The scaling factor is set. cubic meters per second. cubic meters per second. Its hierarchical compression parameters are {algorithm: DCT, quantization step size: 0.05025}.

[0067] Structured compression is performed on the hierarchical representation data according to the hierarchical compression parameters to generate a compressed payload. The LZW algorithm is applied to the data in segment 1 [5.20,…,6.51] to generate a binary compressed data stream, denoted as binary A. The DCT transform is applied to the data in segment 2 [10.1,10.1,12.6,12.5], and quantization is performed using a quantization step size of 0.05025 to generate another binary compressed data stream, denoted as binary B. The compressed payload is the set of binary A and binary B. Finally, the compressed payload is written into the hierarchical description information and associated with the frame fingerprint to generate a hierarchical encapsulation unit. The structure of the constructed hierarchical encapsulation unit is as follows: Header: Frame fingerprint: “c3a2…e9f1”; Hierarchical description information: [{segment index:1, level: critical layer, algorithm:LZW}, {segment index:2, level: non-critical layer, algorithm:DCT, quantization step size:0.05025}]; Body (compressed payload): [binary A, binary B]. This complete data structure is the final generated layered encapsulation unit.

[0068] S6. Generate a verifiable encapsulation packet based on the layered encapsulation unit, execute a hierarchical transmission strategy on the verifiable encapsulation packet based on the link status parameters, generate a transmission log and send it to the cloud; In one embodiment of the present invention, step S6 includes the following steps: Based on the layered encapsulation unit, the layered verification value is calculated, and a verification encapsulation unit is generated; The verification encapsulation unit is divided into packets and sequentially marked to generate a verifiable encapsulation packet; Based on the link state parameters, the verifiable encapsulated packet is subjected to transmission queue shaping to generate a transmission queue; The hierarchical sending strategy is executed according to the sending queue, the sending results are recorded, a sending log is generated, and the log is sent to the cloud. Specifically, after generating the layered encapsulation unit containing compressed data and metadata in step S5, this step aims to increase the transmission robustness of this data unit and execute an intelligent, prioritized transmission strategy based on real-time network conditions to ensure that critical information can reach the cloud efficiently and reliably. The compressed payload of the layered encapsulation unit is naturally divided into critical layer data and non-critical layer data. To achieve more refined error detection, the system independently calculates check values ​​for these two data parts; these are the layered check values. A commonly used check value calculation method is Cyclic Redundancy Check (CRC), such as the CRC-32 algorithm, which can generate a 32-bit short check code from a data block of arbitrary length and has a high detection rate for burst errors occurring during data transmission. The calculated critical layer check values ​​and non-critical layer check values ​​are appended to the header of the layered encapsulation unit, forming a structurally enhanced check encapsulation unit.

[0069] After obtaining the parity encapsulation unit, it needs to be packetized and sequenced to generate a verifiable encapsulation packet. Since network transmission typically has a Maximum Transmission Unit (MTU) limitation, a complete parity encapsulation unit may need to be split into multiple smaller data packets before transmission. The packetization process involves dividing the parity encapsulation unit according to the MTU size. To ensure that the receiving end can correctly reassemble these data packets, each packet is assigned a sequence mark, such as "Packet 1 of 3", "Packet 2 of 3", etc. This set of sequence-marked data packets together constitutes the verifiable encapsulation packet.

[0070] Based on the link state parameters obtained in step S3, the verifiable encapsulated packets undergo transmission queue shaping to generate a transmission queue. Transmission queue shaping is a dynamic priority sorting process, its core purpose being to ensure that packets containing critical layer data are transmitted preferentially even under poor network conditions. The system calculates a transmission priority score for each packet in the verifiable encapsulated packet, a score that depends on both the importance of the packet content and the current network conditions. Priority Score The calculation formula is as follows: , In this formula, It is a data packet The priority score determines the priority of the data transmission; the higher the score, the faster the data is transmitted. It is a data packet The base level weight, if the data packet Data containing critical layer segments is assigned a higher value, such as 100; data containing only non-critical layer segments is assigned a lower value, such as 10. The weighting is based on the difference in importance between critical and non-critical business data in the application. Based on the analysis of 200 sets of historical industrial sensor measurement data, a 10-fold weighting difference can ensure the priority transmission of critical data in most network fluctuation scenarios. It is a transmission constraint indicator obtained from step S3, which comprehensively reflects the current network and device status. yes The theoretical maximum value, therefore This item can be seen as "network degradation"; the worse the network, the larger this value. This is an adjustment factor used to control the degree to which network conditions affect priority; its value can be set to 50 to strike a balance between network quality and data importance. All data packets are processed according to... The values ​​are sorted from high to low to form a sending queue.

[0071] Finally, the hierarchical sending strategy is executed based on the generated sending queue, and the sending results are recorded. A sending log is generated and sent to the cloud. Executing the sending queue means sending data packets sequentially according to the queue order. The hierarchical sending strategy also includes an adaptive dropping mechanism: the system sets a transmission constraint threshold. When real-time transmission constraint indicators When the network speed falls below this threshold, it indicates extremely poor network conditions. In this case, the system will only send data packets with a priority score higher than a certain threshold (usually packets containing critical layer data) and actively discard low-priority packets to ensure the transmission of core information. Every sending operation, including sending time, packet sequence number, and sending status, will be recorded in detail to form a sending log. This sending log will eventually be sent to the cloud for subsequent link performance analysis and strategy optimization.

[0072] For example, following the example from step S5, a layered encapsulation unit is known to have been generated, with a frame fingerprint of "c3a2…e9f1". The compressed payload contains binary A of the critical layer and binary B of the non-critical layer. Meanwhile, the current transmission constraint metrics are known from step S3. First, calculate the layered checksum based on the layered encapsulation unit. Apply the CRC-32 algorithm to the critical layer payload binary A to obtain the checksum 0x1234ABCD. Apply the CRC-32 algorithm to the non-critical layer payload binary B to obtain the checksum 0x5678EF01. Add these two checksums to the unit header to generate the checksum encapsulation unit. Assuming the total size of this unit is 2000 bytes, while the network MTU is 1500 bytes, it needs to be split into two data packets. Data packet 1: Contains the original header, two checksums, the complete binary A, and the first half of binary B, and is marked as "Packet 1 of 2". Data packet 2: Contains the second half of binary B and is marked as "Packet 2 of 2". These two data packets together constitute the verifiable encapsulation packet.

[0073] Subsequently, send queue shaping is performed on the verifiable encapsulated packets. Base-level weights are then set. , Set the adjustment coefficient. Theoretical maximum value of transmission constraint index Data packet 1 contains critical layer data, and its priority score is calculated as follows: Data packet 2 contains only non-critical layer data, and its priority score is calculated as follows: .because The generated sending queue is [data packet 1, data packet 2]. Finally, a tiered sending strategy is executed based on the sending queue. A transmission constraint threshold is set. Because currently The network conditions are acceptable, and the system will send both data packets in full according to the queue order. Simultaneously, a sending log will be generated, which may contain the following: {Frame fingerprint: "c3a2…e9f1", Sending record: [{Timestamp:…, Packet sequence number: 1 / 2, Status: Success}, {Timestamp:…, Packet sequence number: 2 / 2, Status: Success}]}. This sending log will then be sent to the cloud.

[0074] S7. Receive the verifiable encapsulated packet and perform integrity verification according to the verification encapsulation unit to generate a verification result; when the verification result indicates that the verification passed, decapsulate the verifiable encapsulated packet, generate a layered payload, and perform decompression and recovery on the layered payload to generate a recovery monitoring data frame; when the verification result indicates that the verification failed, perform error location on the verifiable encapsulated packet, generate error location information, and form a retransmission request; perform availability assessment on the recovery monitoring data frame and generate the policy feedback information based on the link statistics results, and send it to the edge. Specifically, this step is the cloud-based receiving and feedback link in the entire data transmission and processing closed loop. Its core tasks are to verify the integrity of the received data, recover and reconstruct valid data, and generate feedback information based on the recovery quality and statistical data of the transmission process to guide the adjustment of edge strategies.

[0075] The process begins with the cloud receiving a verifiable encapsulation packet from the edge. Since the verifiable encapsulation packet may be split into multiple data packets for transmission, the cloud first buffers and reassembles the received data packets according to the sequence markers within the packet until a complete verification encapsulation unit is restored. After restoration, the cloud performs integrity verification based on the verification encapsulation unit. The header of the verification encapsulation unit contains layered verification values ​​for critical and non-critical layers. The cloud recalculates the verification values ​​for the corresponding critical and non-critical layer data in the received layered payload using the same verification algorithm (e.g., CRC-32) as the edge. Subsequently, the newly calculated verification value is compared with the original verification value recorded in the encapsulation unit header. If the comparison results match, a verification result indicating successful verification is generated; otherwise, a verification result indicating failed verification is generated.

[0076] When the verification result indicates that the verification has passed, the system enters the data recovery process. First, the verifiable package is decapsulated, i.e., the header information of the verification package unit is parsed to extract the hierarchical description information and layered payloads. The hierarchical description information specifies the compression algorithm and related parameters used for each data segment. Next, the system performs decompression recovery on the layered payloads. Based on the hierarchical description information, the corresponding lossless decompression algorithm (e.g., inverse LZW algorithm) is applied to the data belonging to the critical layer to accurately recover it. For the data belonging to the non-critical layer, the corresponding lossy decompression algorithm (e.g., inverse discrete cosine transform) is applied, and parameters such as the quantization step size recorded in the header are used to recover it as approximate data. Finally, all the recovered data segments are recombined according to the original field structure to generate a recovery monitoring data frame.

[0077] When the verification result indicates a failure, the system initiates an error handling process. Because verification is performed layer by layer, the system can locate errors within verifiable packets. For example, if the checksum of critical layer data matches but the checksum of non-critical layer data does not, the system can determine that the error occurred in the transmission of non-critical layer data. This location information, i.e., error location information, is recorded. Based on this information, the system generates a precise retransmission request, such as requesting the retransmission of only the erroneous non-critical layer data, rather than the entire data frame, thereby improving communication efficiency.

[0078] Whether recovery is successful or a retransmission request is requested, the final step is to generate feedback information. The system performs an availability assessment on the successfully generated recovery monitoring data frames. The availability assessment aims to quantify the quality of the recovered data, especially for non-critical layer data that has undergone lossy compression. Data availability score. It can be calculated using a function related to compression parameters:

[0079] In this formula, Non-critical layer fragments The data availability score ranges from 0 to 1, with higher values ​​indicating better data quality. It is the quantization step size obtained during decapsulation and used for this segment. This is a dimensionless sensitivity coefficient used to adjust the impact of the quantization step size on the availability score. Its value is set based on the business's sensitivity to data accuracy. For example, for a hydrological model with high accuracy requirements, based on simulation analysis of 200 sets of industrial sensor measured data, the value is set... A score of 20 effectively reflects changes in availability. For critical layer segments requiring lossless recovery, the availability score remains constant at 1. The system also incorporates link statistics analyzed from transmission logs received from the edge, such as packet loss rate and retransmission count. Finally, the data availability assessment results are integrated with the link statistics to generate policy feedback information, which is then sent to the edge, providing a basis for the adaptive policy adjustment in step S8.

[0080] For example, following the example of step S6, the cloud receives two data packets related to the frame fingerprint “c3a2…e9f1”, labeled as “Packet 1 of 2” and “Packet 2 of 2”.

[0081] The cloud reassembles these two data packets into a complete checksum encapsulation unit. The critical layer checksum recorded in the unit header is 0x1234ABCD, and the non-critical layer checksum is 0x5678EF01. The cloud then recalculates the CRC-32 checksum for the received payload (binary A and binary B).

[0082] Scenario 1: Verification Passed. The calculated new verification value is completely consistent with the recorded value. The system generates the verification result: "Verification Passed". Next, the system decapsulates the verifiable package and reads the hierarchical description information: Fragment 1 uses LZW lossless compression, fragment 2 uses DCT lossy compression, and the quantization step size... The system performs inverse LZW decompression on binary A to recover the water level data of the critical layer. It then performs inverse DCT decompression on binary B to recover the flow velocity data of the non-critical layers. These two sets of data are then merged to generate a recovery monitoring data frame.

[0083] Scenario 2: Verification Failure. Assume binary B is corrupted during transmission, and the calculated new verification value is 0xABCDEF98, which does not match the recorded 0x5678EF01. The system generates the verification result: "Verification Failed." The system performs error localization, finding that critical layer verification passed, but non-critical layer verification failed, generating the error localization information: "Non-critical layer data corrupted." Based on this, a retransmission request is generated: "Request to retransmit non-critical layer data of frame c3a2…e9f1."

[0084] After successful recovery in Scenario 1, the system performs an availability assessment. A sensitivity coefficient is then set. Usability score of key layer fragment 1 Availability score of non-critical layer fragment 2 Meanwhile, cloud analysis of the received transmission logs revealed no packet loss or retransmission records, with a link statistics result of 0% packet loss rate. Finally, the system generates policy feedback information: {Frame Fingerprint: "c3a2…e9f1", Availability Assessment: {Fragment 1:1, Fragment 2:0.4988}, Link Statistics: {Packet Loss Rate: 0.0}}, and sends this information to the edge.

[0085] S8. Receive and parse the verification results and policy feedback information returned by the cloud, generate policy update quantity, and update the generation rules of the compression control vector based on the policy update quantity.

[0086] In one embodiment of the present invention, step S8 includes the following steps: The verification result returned from the cloud is analyzed for consistency with the policy feedback information to generate a feedback feature vector; The feedback feature vector is correlated with the sending log to generate bottleneck attribution results; The strategy update amount is calculated based on the bottleneck attribution results, the generation rule of the compression control vector is updated, and update confirmation information is output.

[0087] Specifically, such as Figure 4 As shown, this step constitutes the closed-loop feedback and intelligent optimization link of the entire adaptive data compression and transmission system. After receiving the execution results returned from the cloud, the edge device diagnoses system bottlenecks through in-depth analysis and dynamically adjusts its core control strategy to continuously adapt to changing data characteristics and network environment. This process begins with the edge device receiving the verification results and policy feedback information returned from the cloud and performing consistency parsing to generate a feedback feature vector. The policy feedback information issued in step S7 includes data availability assessment and link statistics results, while the verification results directly indicate the success or failure of data transmission. Consistency parsing transforms this heterogeneous information into a standardized numerical vector that can be used for quantitative analysis. Feedback Feature Vector The composition is as follows: , In this vector, It is the arithmetic mean of the availability scores of all data fragments extracted from the strategy feedback information, which comprehensively reflects the quality of the data after it is recovered in the cloud. The packet loss rate, obtained from link statistics, directly measures the reliability of network transmission. It is a quantitative representation of the verification result; for example, a pass is 1, and a fail (partial or complete) is 0. This feedback feature vector provides quantitative input for subsequent attribution analysis.

[0088] The feedback feature vector is correlated with the sending logs stored locally at the edge to generate bottleneck attribution results. The sending logs record the specific actions during data transmission, while the feedback feature vector reflects the final effect of those actions. The correlation analysis uses a set of pre-defined logical rules to determine the root cause of system performance bottlenecks. For example: if Below the preset usability target (e.g., 0.9), while Extremely low values ​​indicate that the data transmission process itself is reliable, but the recovered data quality is poor, likely due to excessive compression. In this case, the bottleneck is attributed to "over-compression." A packet loss rate exceeding a preset threshold (e.g., 0.05) directly indicates poor network conditions, leading to data loss. In this case, the bottleneck is attributed to "network congestion." A value of 0 indicates data verification failure, which is usually related to severe data corruption caused by network congestion. In this case, the bottleneck attribution result is also attributed to "network congestion". If all indicators are within the ideal range, the bottleneck attribution result is "optimal state".

[0089] The strategy update amount is calculated based on the generated bottleneck attribution results and used to update the generation rule of the compressed control vector defined in step S3. Then, update confirmation information is output. The core of the strategy update is adjusting the compressed control vector generation formula. Global scaling factor Strategy update volume The calculation logic is as follows: , Here, It is a fixed adjustment step size, and its value is set based on closed-loop simulation tests using 200 sets of industrial sensor measurement data. Setting it to 0.5 can ensure convergence speed while avoiding strategy oscillation. When attributed to "over-compression," the system needs to reduce the compression intensity, therefore reducing... When the cause is attributed to "network congestion," the system needs to send less data to accommodate the network, thus increasing the compression intensity, i.e., increasing the compression strength. Value. Updated scaling factor. for: , in These are the coefficients before the update. (Use...) Function ensures The value will not become negative. This updated... This new value will replace the old value and will be used to generate the compressed control vector for all subsequent monitoring data frames. After the update is complete, the system will record an update confirmation message to track the evolution of the strategy.

[0090] For example, following the example from step S7, the edge device receives policy feedback information returned from the cloud: {Frame Fingerprint: "c3a2…e9f1", Availability Assessment: {Fragment 1: 1, Fragment 2: 0.4988}, Link Statistics: {Packet Loss Rate: 0.0}}, and the verification result: "Verification passed". First, consistency parsing is performed to generate a feedback feature vector and average availability. Packet loss rate The verification result is "passed", therefore The generated feedback feature vector The result is [0.7494, 0.0, 1].

[0091] Correlation analysis was performed between the feedback feature vector and the transmission log. Edge queries of the transmission log corresponding to the frame fingerprint "c3a2…e9f1" revealed that all data packets were successfully transmitted on the first attempt. The availability target was set to 0.9. Currently... And packet loss rate The value is far below the threshold. This situation meets the "overcompression" criterion. Therefore, the generated bottleneck attribution result is "overcompression". Finally, the policy update amount is calculated based on the bottleneck attribution result, and the rules are updated. Since the attribution is "overcompression", the policy update amount... In step S3, the global scaling factor... The initial value is 10. Calculate the new coefficients: The system will use its internal global scaling factor. Updated to 9.5. Step S3 will use this new version when processing the next monitoring data frame. The compression control vector is calculated using the value, thereby appropriately reducing the compression intensity of non-critical layer data. At the same time, the system outputs an update confirmation message: "Policy update complete: Global scaling factor adjusted from 10.0 to 9.5, attributed to: over-compression".

[0092] Based on the same inventive concept, such as Figure 5 As shown, the present invention also provides a method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration, wherein the system includes an edge terminal and a cloud terminal; wherein, as Figure 6 As shown, the edge includes: The data framing module is used to acquire the raw monitoring data output by the water conservancy monitoring source, perform time alignment and data cleaning, obtain monitoring data frames, calculate the frame-level summary information of the monitoring data frames, and generate frame fingerprints. The importance assessment module is used to parse the monitoring data frame and divide it into segments according to data type and change characteristics to obtain a segment set, and to extract features from the segment set to generate a segment importance vector; The constraint fusion module is used to obtain link state parameters and edge resource state parameters, fuse them to generate a transmission constraint vector, and fuse the transmission constraint vector with the segment importance vector to generate a compression control vector. The hierarchical expression module is used to hierarchically divide the fragment set according to the fragment importance vector, generate a hierarchical fragment set, and construct hierarchical expression data based on the compression control vector and the hierarchical fragment set. The compression and encapsulation module is used to perform structured compression on the layered representation data and perform layered encapsulation in combination with the frame fingerprint to generate a layered encapsulation unit; The verification and sending module is used to generate a verification and encapsulation unit based on the hierarchical encapsulation unit, generate a verifiable encapsulation packet based on the verification and encapsulation unit, execute a hierarchical sending strategy on the verifiable encapsulation packet based on the link state parameters, generate a sending log and send it to the cloud. The policy update module is used to receive and parse the verification results and policy feedback information returned by the cloud, generate a policy update amount, and update the generation rules of the compression control vector based on the policy update amount. The cloud includes: The verification and decapsulation module is used to receive the verifiable package and perform integrity verification according to the verification and encapsulation unit to generate a verification result; when the verification result indicates that the verification is passed, the verifiable package is decapsulated to generate a layered payload and the layered payload is decompressed and restored to generate a recovery monitoring data frame. The error location module is used to locate errors in the verifiable package when the verification result indicates that the verification fails, generate error location information and form a retransmission request. The feedback generation module is used to perform availability assessment on the recovery monitoring data frame and generate the policy feedback information by combining the link statistics results, and then send it to the edge.

[0093] All equivalent changes and modifications made in accordance with the teachings of this invention are still within the scope of this invention. Those skilled in the art will readily conceive of other embodiments of this invention upon considering the specification and the disclosure of practical truth. This application is intended to cover any variations, uses, or adaptations of this invention that follow the general principles of this invention and include common knowledge or conventional techniques in the art not described herein.

Claims

1. A method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration, characterized in that, The method includes: The raw monitoring data output from the water conservancy monitoring source is obtained, time alignment and data cleaning are performed to obtain monitoring data frames, and the frame-level summary information of the monitoring data frames is calculated to generate frame fingerprints. The monitoring data frame is parsed and divided into segments according to data type and change characteristics to obtain a segment set. Feature extraction is performed on the segment set to generate a segment importance vector. The link status parameters and edge resource status parameters are obtained and fused to generate a transmission constraint vector. The transmission constraint vector is then fused with the segment importance vector to generate a compression control vector. The fragment set is hierarchically divided according to the fragment importance vector to generate a hierarchical fragment set, and hierarchical expression data is constructed based on the compression control vector and the hierarchical fragment set; The hierarchical representation data is subjected to structured compression, and the frame fingerprint is combined with hierarchical encapsulation to generate hierarchical encapsulation units; A verifiable encapsulation packet is generated based on the layered encapsulation unit. A hierarchical transmission strategy is executed on the verifiable encapsulation packet based on the link state parameters. A transmission log is generated and sent to the cloud. The system receives and parses the verification results and policy feedback information returned from the cloud, generates a policy update quantity, and updates the generation rules of the compression control vector based on the policy update quantity.

2. The method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration according to claim 1, characterized in that, The process of acquiring raw monitoring data output from water conservancy monitoring sources, performing time alignment and data cleaning to obtain monitoring data frames, calculating frame-level summary information of the monitoring data frames, and generating frame fingerprints includes: Obtain the sampling time identifier and data fields of each water conservancy monitoring source to generate raw monitoring data; The original monitoring data is time-aligned and missing fields are filled with consistency to generate an aligned data sequence. Noise suppression and outlier screening are performed on the aligned data sequence to construct a monitoring data frame; Frame-level summary information is extracted from the monitoring data frame and the summary is encoded to generate a frame fingerprint.

3. The method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration according to claim 2, characterized in that, The process of parsing the monitoring data frame and dividing it into segments according to data type and change characteristics to obtain a segment set, and then extracting features from the segment set to generate a segment importance vector includes: Obtain the field structure and field semantic identifiers of the monitoring data frame, and generate field structure information; The monitoring data frame is segmented based on the field structure information, and the segment boundaries are recorded to generate a segment set; Extract change intensity features and stability features from the fragment set to generate a change feature set; The importance of the set of changing features is evaluated to obtain the fragment importance vector.

4. The method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration according to claim 3, characterized in that, The process of fusing the obtained link state parameters and edge resource state parameters to generate a transmission constraint vector, and then fusing the transmission constraint vector with the segment importance vector to generate a compression control vector, includes: Obtain link status parameters and perform time-series aggregation to generate link status feature vectors; Obtain edge resource status parameters and perform load characterization to generate resource status feature vectors; The link state feature vector and the resource state feature vector are fused to generate a transmission constraint vector; The transmission constraint vector and the fragment importance vector are jointly mapped to generate a compression control vector.

5. The method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration according to claim 4, characterized in that, The step of hierarchically dividing the fragment set according to the fragment importance vector to generate a hierarchical fragment set, and constructing hierarchical representation data based on the compression control vector and the hierarchical fragment set includes: The fragment set is hierarchically divided according to the fragment importance vector to generate a hierarchical fragment set containing key layer fragments and non-key layer fragments; Fidelity representation construction is performed on the key layer fragments in the hierarchical fragment set to generate key layer representation data; Approximate representation construction is performed on the non-critical layer fragments in the hierarchical fragment set while preserving trend description information, generating non-critical layer representation data; The key layer expression data and the non-key layer expression data are merged to generate hierarchical expression data.

6. The method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration according to claim 5, characterized in that, The step of performing structured compression on the hierarchical representation data and combining it with the frame fingerprint for hierarchical encapsulation to generate a hierarchical encapsulation unit includes: Obtain the hierarchical description information of the hierarchical representation data and select the compression path to generate hierarchical compression parameters; The hierarchical representation data is subjected to structured compression based on the hierarchical compression parameters to generate a compressed payload; The compressed payload is written into the hierarchical description information and associated with the frame fingerprint to generate a hierarchical encapsulation unit.

7. The method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration according to claim 6, characterized in that, The step of generating a verifiable encapsulation packet based on the layered encapsulation unit, executing a hierarchical transmission strategy on the verifiable encapsulation packet based on the link state parameters, generating transmission logs, and sending them to the cloud includes: Based on the layered encapsulation unit, the layered verification value is calculated, and a verification encapsulation unit is generated; The verification encapsulation unit is divided into packets and sequentially marked to generate a verifiable encapsulation packet; Based on the link state parameters, the verifiable encapsulated packet is subjected to transmission queue shaping to generate a transmission queue; The hierarchical sending strategy is executed according to the sending queue, the sending results are recorded, a sending log is generated and sent to the cloud.

8. The method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration according to claim 7, characterized in that, The method further includes a cloud processing step, which includes: Receive the verifiable package and perform integrity verification according to the verification package unit to generate a verification result; When the verification result indicates that the verification is passed, the verifiable package is decapsulated to generate a layered payload and the layered payload is decompressed and restored to generate a recovery monitoring data frame. When the verification result indicates that the verification fails, error location is performed on the verifiable package, error location information is generated, and a retransmission request is formed. An availability assessment is performed on the recovery monitoring data frame, and the policy feedback information is generated by combining the link statistics results and sent to the edge.

9. The method for compressing and transmitting water conservancy monitoring data based on edge-cloud collaboration according to claim 8, characterized in that, The process of receiving and parsing the verification results and policy feedback information returned from the cloud, generating a policy update quantity, and updating the generation rule of the compression control vector based on the policy update quantity includes: The verification result returned from the cloud is analyzed for consistency with the policy feedback information to generate a feedback feature vector; The feedback feature vector is correlated with the sending log to generate bottleneck attribution results; The strategy update amount is calculated based on the bottleneck attribution results, the generation rule of the compression control vector is updated, and update confirmation information is output.

10. A water conservancy monitoring data compression and transmission system based on edge-cloud collaboration, characterized in that, The system includes an edge and a cloud; wherein, the edge includes: The data framing module is used to acquire the raw monitoring data output by the water conservancy monitoring source, perform time alignment and data cleaning, obtain monitoring data frames, calculate the frame-level summary information of the monitoring data frames, and generate frame fingerprints. The importance assessment module is used to parse the monitoring data frame and divide it into segments according to data type and change characteristics to obtain a segment set, and to extract features from the segment set to generate a segment importance vector; The constraint fusion module is used to obtain link state parameters and edge resource state parameters, fuse them to generate a transmission constraint vector, and fuse the transmission constraint vector with the segment importance vector to generate a compression control vector. The hierarchical expression module is used to hierarchically divide the fragment set according to the fragment importance vector, generate a hierarchical fragment set, and construct hierarchical expression data based on the compression control vector and the hierarchical fragment set. The compression and encapsulation module is used to perform structured compression on the layered representation data and perform layered encapsulation in combination with the frame fingerprint to generate a layered encapsulation unit; The verification and sending module is used to generate a verification and encapsulation unit based on the hierarchical encapsulation unit, generate a verifiable encapsulation packet based on the verification and encapsulation unit, execute a hierarchical sending strategy on the verifiable encapsulation packet based on the link state parameters, generate a sending log and send it to the cloud. The policy update module is used to receive and parse the verification results and policy feedback information returned by the cloud, generate a policy update amount, and update the generation rules of the compression control vector based on the policy update amount. The cloud includes: The verification and decapsulation module is used to receive the verifiable package and perform integrity verification according to the verification and encapsulation unit to generate a verification result; when the verification result indicates that the verification is passed, the verifiable package is decapsulated to generate a layered payload and the layered payload is decompressed and restored to generate a recovery monitoring data frame. The error location module is used to locate errors in the verifiable package when the verification result indicates that the verification fails, generate error location information and form a retransmission request. The feedback generation module is used to perform availability assessment on the recovery monitoring data frame and generate the policy feedback information by combining the link statistics results, and then send it to the edge.

Images