Iot device data lightweight transmission method based on edge computing

Abstract

The application discloses an edge-computing-based lightweight transmission method for Internet of Things device data, and particularly relates to the technical field of data transmission, which comprises the following steps: step one, a dynamic sliding window is established at an edge node, a deviation between a new data point and a historical trend is calculated, a data source is filtered by combining an adaptive threshold dynamically adjusted based on historical fluctuations and network conditions, and an optimization mechanism based on false negative rate and false positive rate feedback is introduced to correct a weight coefficient; step two, a device relationship graph is maintained to identify spatially related sensor groups, spatial redundancy analysis is performed on key data points in the groups, and a representative point or statistical feature uploading mode is adopted for data fusion; and step three, data packets are evaluated in terms of business priority, information entropy and timeliness, and the three-level linkage mechanism effectively eliminates data redundancy in the time and space dimensions, realizes fine allocation of network resources, and greatly reduces uplink data volume and bandwidth occupation.

Description

Technical Field

[0001] This invention relates to the field of data transmission technology, and more specifically, to a lightweight data transmission method for Internet of Things (IoT) devices based on edge computing. Background Technology

[0002] With the rapid development and widespread application of IoT technology, hundreds of millions of sensors and devices continuously generate massive amounts of data, posing severe challenges to network transmission and cloud processing capabilities. In the traditional "edge-to-cloud" direct connection transmission architecture, all raw data is directly uploaded to the cloud, which not only consumes a large amount of network bandwidth, leading to transmission delays and network congestion, but also places a heavy storage and computing burden on cloud data centers. To alleviate this problem, edge computing technology has emerged, which deploys computing nodes at the network edge close to the data source to preprocess the data, thereby achieving lightweight data transmission. However, existing edge computing lightweighting methods are usually relatively simple, such as using fixed-period aggregation reporting or static threshold-triggered reporting, which fail to fully consider the temporal correlation, spatial correlation, and differences in the value density of the data itself, resulting in limited lightweighting effects or significant accuracy loss during data restoration.

[0003] At the data acquisition level, existing technologies mostly employ fixed-frequency sampling or simple threshold filtering, making it difficult to effectively identify the temporal correlation of data. For example, while fixed-period reporting can reduce the amount of data, it cannot capture brief abnormal peaks within the period, leading to the loss of critical events. Static threshold-triggered reporting, when facing slow-drifting faults, generates a large amount of "normal" but "abnormal" data, causing misjudgments or missed diagnoses. Furthermore, in multi-sensor deployment scenarios, existing solutions mostly process data from each sensor independently, failing to fully consider the spatial redundancy of data from multiple sensors within the same area. For example, multiple temperature sensors in the same warehouse may have highly consistent readings, yet all are uploaded, resulting in severe spatial data redundancy. Simultaneously, at the data transmission level, existing technologies typically use the same transmission strategy for all data, failing to differentiate processing based on the data's business value, information content, and timeliness. This leads to high-value critical data potentially being lost or delayed due to low-value data such as heartbeat logs consuming bandwidth during network congestion.

[0004] To address the problems existing in the above-mentioned technologies, there is an urgent need for an intelligent and lightweight method that can achieve full-link intelligent management from data acquisition and data processing to data transmission. This method would accurately identify and retain key information, eliminate data redundancy in the time and spatial dimensions, and achieve fine-grained allocation of network resources based on data value. In this way, while ensuring data integrity and business needs, network bandwidth consumption and cloud storage pressure can be reduced to the greatest extent. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a lightweight data transmission method for Internet of Things (IoT) devices based on edge computing.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A lightweight data transmission method for IoT devices based on edge computing includes the following steps:

[0008] Step 1: Establish a dynamic sliding window for IoT devices on the edge computing node. By calculating the deviation between new data points and historical trends, and combining an adaptive threshold that is dynamically adjusted based on historical fluctuations and network conditions, perform preliminary filtering of data sources. Only data points that have undergone meaningful changes or mutations are marked as key data points and stored in the queue to be uploaded.

[0009] Step 2: The edge node maintains the device relationship map, identifies spatially related sensor groups, and when multiple sensors in a group generate key data points within a similar time window, performs spatial redundancy analysis, and fuses the data that meet the spatial redundancy conditions to generate representative point data or statistical feature data, replacing the original multiple key data points.

[0010] Step 3: The edge node evaluates the value of the data packets to be uploaded, calculates the total value score from three dimensions: business priority, information entropy, and timeliness, and classifies them into different value levels accordingly. Different compression algorithms and transmission protocols are used to upload data packets of different levels.

[0011] Specifically, in step one:

[0012] The dynamic sliding window includes a main sampling window and an auxiliary validation window. The main sampling window is used for trend analysis and its size is dynamically adjusted according to the stability of the data. The auxiliary validation window has a fixed size and is used for mutation detection.

[0013] The adaptive threshold is dynamically calculated based on the historical data fluctuation range and the current network congestion status. The historical fluctuation weight and network congestion weight coefficient are periodically corrected through an optimization mechanism based on feedback from the false negative rate and false positive rate.

[0014] Specifically, the optimization mechanism based on feedback of false negative and false positive rates includes:

[0015] Edge nodes periodically retrieve the number of missed reports and false reports from the cloud in the previous period, and calculate the missed report rate and false report rate.

[0016] Set target false alarm rate and target false alarm rate, and use proportional control law to calculate the adjustment of weight coefficients;

[0017] The weighting coefficients are updated based on the adjustment amount, and the updated coefficients are limited to a preset range for use in the adaptive threshold calculation of the next cycle.

[0018] Specifically, the automatic construction method of the device relationship map in step two includes:

[0019] In the early stages of system deployment, edge nodes collect raw data and report it to the cloud. The cloud uses the Pearson correlation coefficient to calculate the correlation between the data sequences of any two sensors. If the absolute value of the correlation coefficient is greater than a preset threshold, the two are determined to be highly correlated and classified into the same correlation group.

[0020] The map supports periodic dynamic updates or manual corrections by administrators.

[0021] Specifically, the spatial redundancy analysis includes:

[0022] When multiple sensors in a group generate key data points within a similar time window, the difference between the readings of each sensor is calculated. If the maximum difference is less than the preset spatial redundancy threshold, spatial redundancy is determined to exist.

[0023] The fusion process includes representative point uploading or statistical feature uploading. Representative point uploading selects sensor data with the highest signal-to-noise ratio or the most sufficient battery power within the group. Statistical feature uploading calculates and uploads the average, median, maximum, or minimum value of the data within the group.

[0024] Specifically, in step three:

[0025] The total value assessment score is calculated using a multi-factor weighted summation model, first by normalizing each dimension:

[0026] Business priorities are mapped to numerical values, information entropy is directly used with the normalized value, and timeliness is calculated based on the time difference between the data generation time and the current time and the maximum validity period of the data.

[0027] Then multiply by the corresponding weighting coefficient and sum to get the total score. Based on the total score, divide the score into three value levels: high, medium, and low.

[0028] Specifically, the weighting coefficients are dynamically optimized through a weighting adjustment mechanism based on historical transmission performance feedback, including:

[0029] Edge nodes periodically obtain the actual utilization rate of each level of data from the cloud in the previous period, and calculate the deviation from the target utilization rate;

[0030] The adjustment amount of each weight coefficient is calculated using the proportional control law. After updating the weight coefficients, they are normalized and limited for use in the next period's value assessment.

[0031] Specifically, the differentiated compression and transmission strategies include:

[0032] For high-value data, lossless compression algorithms are used and the data is transmitted in real time via a reliable TCP connection.

[0033] For medium-value data, a lossy compression algorithm is used and the data is transmitted in batches via the UDP protocol;

[0034] For low-value data, after aggregation and compression at edge nodes, it is transmitted in batches through a non-real-time channel during idle network periods.

[0035] It also includes edge node state preservation mechanisms:

[0036] Specifically, the queue to be uploaded in step one and the data list in step two are persisted using non-volatile storage, and the persistent data is loaded to restore the breakpoint when the edge node restarts;

[0037] Meanwhile, the queue to be uploaded adopts a circular buffer design. When the buffer is full, an alarm is triggered, and low-value data is discarded first to ensure stable system operation.

[0038] The technical effects and advantages of this invention are as follows:

[0039] Through a three-tiered intelligent processing mechanism, a lightweight end-to-end system is achieved, encompassing data acquisition, processing, and transmission. First, at the data acquisition layer, an adaptive filtering technique based on a dynamic sliding window is employed. By calculating the deviation between data points and historical trends in real time and dynamically adjusting thresholds based on network conditions, it accurately identifies and retains trend change points and abrupt shifts, eliminating temporal redundancy at its source. Second, at the data processing layer, spatiotemporal correlation analysis is introduced. A device relationship graph identifies spatially related sensor groups, and representative points or statistical features are fused from highly consistent sensor readings, effectively eliminating spatial redundancy. Finally, at the data transmission layer, a differentiated transmission strategy based on multi-dimensional value assessment is constructed. Data packets are graded according to business priority, information entropy, and timeliness, and uploaded using differentiated compression algorithms and transmission protocols. Simultaneously, a feedback mechanism based on actual utilization is introduced to dynamically optimize evaluation weights, achieving refined allocation of network and computing resources.

[0040] This invention has the following significant advantages: First, through adaptive thresholding and feedback optimization mechanisms, it not only filters out static redundancy but also effectively identifies data trend changes and abrupt changes, achieving source data reduction while ensuring data accuracy. Second, through spatiotemporal correlation analysis, it fuses data packets from multiple related sensors into a single feature data packet, significantly reducing the amount of uplink data, which is particularly suitable for large-scale sensor array deployment scenarios. Third, the hierarchical transmission strategy based on data value ensures that high-value critical data receives the highest priority and most reliable transmission, while low-value data is transmitted or discarded at the lowest cost, maximizing the efficiency of transmission resources. Attached Figure Description

[0041] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] like Figure 1 As shown, the steps of the lightweight data transmission method for IoT devices based on edge computing are as follows:

[0044] Step 1: Adaptive data acquisition and initial filtering. Using a main sampling window and an auxiliary verification window, the deviation of new data points from historical trend lines and the difference from the nearest point are calculated in real time. Combined with an adaptive threshold dynamically calculated based on historical fluctuations and network conditions, it is determined whether the data is redundant and whether to discard it. Simultaneously, an optimization mechanism based on feedback from false negative and false positive rates is introduced to dynamically adjust weighting coefficients, continuously optimizing the threshold and achieving accurate data lightweighting from the source. The specific process is as follows:

[0045] Step 101: On the edge computing node close to the IoT device, establish two dynamic sliding windows for each connected sensor device: a main sampling window. and an auxiliary verification window . Used for routine data collection and trend analysis, the window size can be dynamically adjusted according to the device type and historical data change rate; This is used to quickly verify mutations in the data;

[0046] The main sampling window The initialization size is determined by the device's default sampling frequency at the factory. For example, for a temperature sensor, if the default sampling rate is 1 time / second, then the initialization... A size of 60 represents storing historical data from the last 60 seconds for trend fitting. The logic for "dynamically adjusting" the window size is as follows: if the deviation value... All less than 0.5× This indicates that the data is in a period of extreme stability, at which point the window size should be adjusted. Increase the size to 1.5 times the original size to accommodate more historical data and improve the stability of trend fitting; if a mutation is detected three times consecutively (i.e., meets the condition)... or ,in This represents the deviation between the new data point and the fitted trend line of the historical data in the main window. For adaptive dynamic threshold, The difference between the new data point and the last data point in the auxiliary window. (where the mutation threshold is), then... Reset to initial size for quick response to new changes. Auxiliary window. The size is fixed at 2, and only the latest two data points are retained for mutation detection.

[0047] Step 102: When the new data point x t Upon entry, edge nodes calculate their distance from the main window. The deviation of the trend line fitted to historical data (e.g., using the least squares method). Simultaneously, calculate new data points and auxiliary windows. The difference Δv between the last data point and the last data point.

[0048] Step 103: Set an adaptive threshold This threshold is not a fixed value, but is dynamically calculated based on the historical fluctuation range of the data within the main window (such as the standard deviation σ) and the current network congestion status (obtained from cloud feedback or edge node detection). ,in This refers to the historical volatility weighting coefficient and the weighting coefficient, with values ​​ranging from 1.2 to 2.0, and a default value of 1.5, used to control the sensitivity to historical volatility. This is the network congestion weighting coefficient, with a value between 0.1 and 0.5, and a default value of 0.3. It is used to control the penalty level for network congestion on the threshold. The standard deviation of historical data within the main window (reflecting historical fluctuations). The ratio of current bandwidth to baseline bandwidth reflects network load; when the network is congested (lower ratio), This will lower the threshold, making filtering more stringent and discarding more data to alleviate network pressure. Additionally, the mutation threshold... This is a fixed value related to the sensor hardware, typically set to 5% of the sensor's range or based on the minimum resolution of the analog-to-digital conversion. For example, for a temperature sensor ranging from 0-100°C, It can be set to 2℃ to capture instantaneous jumps.

[0049] In order to make the weighting coefficients and It can continuously optimize based on actual filtering results, and use an optimization mechanism to make corrections, as follows:

[0050] Feedback data collection:

[0051] Edge nodes every fixed period (For example, every hour) Request filtering performance statistics from the cloud for the previous period. The cloud then calculates the following metrics based on actual events (such as actual sensor alarms, manual annotations, or subsequent data verification):

[0052] Number of missed reports The number of events that should have been marked as "critical data points" but were mistakenly identified as redundant and discarded; the number of false alarms. The number of events marked as "critical data points" and uploaded, but which are actually redundant data; total number of decisions. The total number of all data points involved in the filtering decision during this period.

[0053] Performance metric calculation: Calculate the false negative rate for this period. and false alarm rate : , .

[0054] Deviation calculation and adjustment determination: setting the target false negative rate and target false alarm rate (For example, the default values ​​are set according to business needs, with default values ​​of 0.01 and 0.05 respectively). The adjustment amount of the weighting coefficients is calculated using a proportional control law: , ;in Weighting coefficients The adjustment amount, Weighting coefficients The adjustment amount, for Learning step size for The learning step size, ranging from 0.01 to 0.1, is used to control the adjustment speed.

[0055] If the false negative rate is higher than the target, it indicates that the threshold is too high, leading to missed detections, and should be reduced. The threshold should be lowered; if the false alarm rate is higher than the target, it indicates that the threshold is too low, leading to misjudgments, and should be increased. To raise the threshold. Therefore, the adjustment direction is: , ;in For the updated value, Before the update value, for The adjustment amount, For the updated value, Before the update value, for The adjustment amount;

[0056] Parameter limiting and updating: The updated parameters will be used for parameter limiting and updating. and Limiting to a preset range ensures system stability. , ;in =1.2, =2.0, =0.1, =0.5. Edge nodes use the updated value. and Replace the original value for the adaptive threshold calculation in the next cycle.

[0057] Through the above optimization mechanism, the weighting coefficient can be adaptively adjusted according to the false alarms and missed alarms in the actual application scenario, so that the threshold is always kept in the optimal state, further improving the accuracy and robustness of data lightweighting.

[0058] Step 104: Make a filtering decision: if the deviation value Less than the adaptive threshold And the difference It is also less than the mutation threshold. Then the new data point is considered Within the expected trend range, this data is considered redundant and will be discarded locally on the edge node without being uploaded. Conversely, if... or If so, it is considered that the data has undergone a meaningful change or mutation. Mark them as "critical data points" and store them along with their timestamps in the queue to be uploaded.

[0059] Step 2: Spatiotemporal correlation deduplication and fusion. By identifying spatially related sensor groups through a device relationship graph, spatial redundancy analysis is performed when multiple sensors within a group generate key data points within similar time windows. If sensor readings are highly consistent, fusion rules such as representative point uploading or statistical feature uploading are used to merge multiple data points into a single data packet, thereby eliminating spatial data redundancy and significantly reducing the amount of uplink data. The specific process is as follows:

[0060] Step 201: The edge node maintains a device relationship graph, which is automatically constructed or configured by the administrator based on the device's geographical location, function type, or historical data correlation. For example, multiple temperature sensors in the same area are grouped into a correlation group G.

[0061] The automatic construction method for the device relationship graph is as follows: In the initial stage of system deployment (cold start phase), edge nodes collect all raw data at a default frequency and report it to the cloud. The cloud uses the Pearson correlation coefficient to calculate the correlation between the data sequences of any two sensors over the past 24 hours. The calculation formula is: ;in, For sensors and The correlation coefficient of the data series , Sensors and The i-th data point; , Sensors and The average value of a data series;

[0062] If the correlation coefficient If the two sensors are spatially highly correlated, the system automatically assigns them to the same correlation group G. This correlation map supports dynamic updates; edge nodes recalculate the correlation matrix every 24 hours based on locally cached data to adapt to correlation drift caused by device aging or environmental changes. Administrators can also manually correct the map via the northbound interface.

[0063] Step 202: When multiple sensors (S1, S2, ..., Sn) within group G generate "key data points" within a similar time window (e.g., within 1 minute), the edge node initiates spatiotemporal correlation analysis. It calculates the spatial similarity matrix of these data points. For example, it calculates the difference in readings Δs12 between sensors S1 and S2.

[0064] Step 203: Set a spatial redundancy threshold T space If the maximum difference Δs among the readings of all sensors within group G... max Less than T space If we consider that the physical phenomena perceived by these sensors at that moment are highly consistent, then there is spatial redundancy.

[0065] Step 204: Perform data fusion according to predefined fusion rules. The fusion rules are as follows:

[0066] Representative point upload: Select data from a representative sensor in group G (such as the one with the highest signal-to-noise ratio or the most sufficient battery power) and upload it, along with a flag indicating that the data represents the status of the entire group.

[0067] Upload statistical features: Instead of uploading all the raw data, calculate and upload the statistical features of the entire group G, such as the mean, median, maximum, and minimum values. For example, upload "Average temperature of region A = 25.3℃".

[0068] Step 205: Replace the original multiple "key data points" with the data after fusion (representative point data or statistical features), store it in the queue to be uploaded, update the data list, and record the original sensor ID to be fused and the hash value of this fusion operation, so that it can be used for consistency verification in the cloud or for traceability when more refined data is needed.

[0069] Step 3: Value Assessment and Differentiated Transmission. Each data packet is assessed for value across three dimensions: business priority, information entropy, and timeliness. A weighted summation model is used to calculate the total score, categorizing the data into high, medium, and low levels. Different compression algorithms and transmission protocols are applied to data of different levels before uploading. Simultaneously, a feedback mechanism based on actual utilization is introduced to dynamically adjust weighting coefficients, continuously optimizing the value assessment model and achieving refined allocation of network and computing resources. The specific process is as follows:

[0070] Step 301: Before data is sent from the edge node to the cloud, perform a value assessment on each data packet in the queue (which may contain a single key data point or fused feature data). The value assessment is based on one or more of the following dimensions:

[0071] Business Priority P: Priorities are preset based on the application type of the data (e.g., security alarm data is the highest level P1, environmental monitoring data is the medium level P2, and device heartbeat logs are the lowest level P3); Information Entropy E: Calculates the information entropy of the data packet content. For outliers or patterns appearing for the first time, the information entropy is high, and the value is high; for recurring patterns, the information entropy is low, and the value is low; Timeliness T: The time the data was generated; newer data generally has higher value.

[0072] Step 302: Evaluate the total score based on the data value. Data packets are divided into different levels. The calculation uses a multi-factor weighted summation model, first normalizing each dimension:

[0073] Priority factor : Map discrete priority values ​​to numerical values, P1 is mapped to 1.0, P2 to 0.6, and P3 to 0.2.

[0074] Information entropy factor Information entropy Calculated using Shannon's formula, the value ranges from 0 to 1 (if the data packet content is standardized and encoded, the maximum possible entropy is 1). This can be used directly. .

[0075] Timeliness factor Define data freshness. Let the data generation time be... The current time is The maximum validity period of the data is (For example, for real-time control data, the validity period might only be 10 seconds; for log data, it might be 1 hour.) Therefore, the formula for calculating the timeliness factor is: ;in This is the timeliness factor (normalized value); if the data is outdated ( ), .

[0076] final, ,in The total score for data value assessment. , , These are the weighting coefficients, and In a specific security monitoring scenario implementation, the following settings are provided: =0.5, =0.3, =0.2;

[0077] Level classification rules: If If so, it is high-value data; if If it is, then it is considered medium-value data; if If so, it is low-value data.

[0078] For the weighting coefficients of value assessment , , It can dynamically optimize based on actual transmission performance and business needs, utilizing a weight adjustment mechanism based on historical transmission performance feedback; the specific steps are as follows:

[0079] Feedback Data Collection: Edge nodes request transmission performance statistics for the previous period from the cloud at fixed intervals (e.g., every hour). The cloud then compiles the following metrics based on the actual usage of data packets (e.g., whether they are used by business systems, whether alarms are triggered, whether they are viewed by users):

[0080] High-value data utilization The percentage of data packets deemed high-value that are actually used by the business system (e.g., triggering subsequent analysis or alarms); Utilization rate of medium-value data. The percentage of data packets deemed medium value that were actually used; utilization rate of low-value data. The percentage of data packets deemed low-value that are actually used.

[0081] Performance deviation calculation: Set target utilization rate (Can be preset according to business needs, for example, 0.9, 0.5, 0.1 respectively). Calculate the deviation between the actual utilization rate and the target value for each level: ; ; ;in , , The utilization deviations for high, medium, and low-value data are respectively... These are the target utilization rates for high-value data, medium-value data, and low-value data. If the actual utilization rate of a certain level is lower than the target, it indicates that the data value of that level is underestimated, and its corresponding weight should be appropriately increased. If the actual utilization rate is higher than the target, it indicates that the value is overestimated, and its weight should be decreased.

[0082] Weight adjustment calculation: The weight adjustment is calculated using the proportional control law. (Corresponding priority factor) ); (corresponding information entropy factor) ); (Corresponding timeliness factor) );

[0083] in They are respectively , , The step size is the learning step size, ranging from 0.01 to 0.1, used to control the adjustment speed; the negative sign indicates that when the utilization rate is lower than the target (ΔU is negative), the adjustment amount is positive, that is, the weight is increased; otherwise, the weight is decreased. , , Weighting coefficients Adjustment amount, weighting coefficient Adjustment amount and weighting coefficient The adjustment amount;

[0084] Weight Update and Limitation: Update weight coefficients: ; ; ;in , , The updated versions are as follows: , , Value; to ensure the sum of the weights is 1, the updated weights are normalized: ;in For the normalized first Each weighting coefficient For the updated number Each weighting coefficient It is the sum of the three updated weight coefficients;

[0085] At the same time, the weights should be limited to a reasonable range (e.g., 0.1 to 0.8) to avoid extreme values ​​that could lead to an unbalanced assessment. ;in For the final use of the first Each weighting coefficient , These are the minimum allowed weight (0.1) and the maximum allowed weight (0.8), respectively; edge nodes use... Replace the original weighting coefficients for the next period's value assessment.

[0086] Step 303: Implement differentiated compression and transmission strategies for data packets of different levels:

[0087] High-value data: Lossless compression algorithms (such as LZ4) are used to ensure data integrity, and the data is transmitted in real-time or near real-time via a reliable TCP connection. A dedicated QoS channel can be requested if necessary.

[0088] For mid-value data: High-compression-ratio lossy compression algorithms (such as quantization of floating-point numbers or differential coding) are employed to maximize compression ratio within acceptable precision loss limits. Batch transmission is performed via UDP, with sequence numbers and checksums added to data packets, allowing for simple packet loss retransmission or error correction in the cloud.

[0089] Low-value data: Perform higher-level aggregation at edge nodes. For example, package all low-value heartbeat logs from the past hour into a single file, compress it using a general compression tool (such as gzip), and then transmit it in batches via a non-real-time channel during periods of network idle time (such as early morning). If local storage is limited, some expired low-value data can even be discarded directly.

[0090] Step 304: The cloud receiver uses the corresponding decompression and parsing algorithm to restore the data according to the policy identifier in the data packet header.

[0091] To ensure that the edge node's state is not lost after a power outage or restart, the upload queue in step 104 and the data list in step 205 are persisted using non-volatile storage (such as Flash or an embedded database). When the edge node starts up, it first loads the persisted data to recover from the breakpoint. Meanwhile, to prevent the upload queue from growing uncontrollably and causing memory overflow, a circular buffer design is used. When the buffer is full, an alarm is triggered, and some low-value data is forcibly discarded according to the FIFO principle (prioritizing the discarding of low-value categories of data defined in step 303), ensuring stable system operation.

[0092] The above formulas are all dimensionless calculations. Dimensionless calculations can be performed using various methods such as standardization, which will not be elaborated here. The formulas are derived from software simulations based on a large amount of collected data, and the preset parameters in the formulas can be set by those skilled in the art according to the actual situation.

[0093] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, ATA hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state ATA hard disk.

[0094] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0095] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0097] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0098] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0099] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable ATA hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A lightweight data transmission method for IoT devices based on edge computing, characterized in that, Includes the following steps: Step 1: Establish a dynamic sliding window for IoT devices on the edge computing node. By calculating the deviation of new data points from historical trends, and combining it with an adaptive threshold that is dynamically adjusted based on historical fluctuations and network conditions, perform preliminary filtering of data sources. Only data points that deviate from historical trends beyond the adaptive threshold or that undergo numerical mutations are marked as key data points and stored in the queue to be uploaded. Step 2: Maintain the device relationship map at the edge nodes, identify spatially related sensor groups, and when multiple sensors in a group generate key data points within a similar time window, perform spatial redundancy analysis, and fuse the data that meet the spatial redundancy conditions to generate representative point data or statistical feature data to replace the original multiple key data points. Step 3: The edge node evaluates the value of the data packets to be uploaded, calculates the total value score from three dimensions: business priority, information entropy, and timeliness, and classifies them into different value levels accordingly. Different compression algorithms and transmission protocols are used to upload data packets of different levels. In step one: The dynamic sliding window includes a main sampling window and an auxiliary validation window. The main sampling window is used for trend analysis and its size is dynamically adjusted according to the stability of the data. The auxiliary validation window has a fixed size and is used for mutation detection. The adaptive threshold is dynamically calculated based on the historical data fluctuation range and the current network congestion status. The historical fluctuation weight and network congestion weight coefficient are periodically corrected through an optimization mechanism based on the feedback of the false negative rate and false positive rate. The optimization mechanism based on feedback of false negative rate and false positive rate includes: Edge nodes periodically retrieve the number of missed reports and false reports from the cloud in the previous period, and calculate the missed report rate and false report rate. Set target false alarm rate and target false alarm rate, and use proportional control law to calculate the adjustment of weight coefficients; The weight coefficients are updated based on the adjustment amount, and the updated coefficients are limited to a preset range for use in the adaptive threshold calculation of the next cycle. The automatic construction method of the device relationship map in step two includes: In the early stages of system deployment, edge nodes collect raw data and report it to the cloud. The cloud uses the Pearson correlation coefficient to calculate the correlation between the data sequences of any two sensors. If the absolute value of the correlation coefficient is greater than a preset threshold, the two are determined to be highly correlated and classified into the same correlation group. The map supports periodic dynamic updates or manual corrections by administrators; The spatial redundancy analysis includes: When multiple sensors in a group generate key data points within a similar time window, the difference between the readings of each sensor is calculated. If the maximum difference is less than the preset spatial redundancy threshold, spatial redundancy is determined to exist. The fusion process includes representative point uploading or statistical feature uploading. Representative point uploading selects sensor data with the highest signal-to-noise ratio or the most sufficient battery power within the group. Statistical feature uploading calculates and uploads the average, median, maximum, or minimum value of the data within the group. After the fusion process is completed, a list of data to be uploaded is generated to record the correspondence between the fused data and the original sensor information; In step three: The total value assessment score is calculated using a multi-factor weighted summation model, first by normalizing each dimension: Business priorities are mapped to numerical values, information entropy is directly used with the normalized value, and timeliness is calculated based on the time difference between the data generation time and the current time and the maximum validity period of the data. Then multiply by the corresponding weighting coefficient and sum to get the total score. Based on the total score, divide into three value levels: high, medium, and low. The weighting coefficients are dynamically optimized through a weighting adjustment mechanism based on historical transmission performance feedback, including: Edge nodes periodically obtain the actual utilization rate of each level of data from the cloud in the previous period, and calculate the deviation from the target utilization rate; The adjustment amount of each weight coefficient is calculated using the proportional control law. After updating the weight coefficients, they are normalized and limited for use in the next period's value assessment.

2. The lightweight data transmission method for IoT devices based on edge computing according to claim 1, characterized in that, The differentiated compression algorithms and transmission protocols include: For high-value data, lossless compression algorithms are used and the data is transmitted in real time via a reliable TCP connection. For medium-value data, a lossy compression algorithm is used and the data is transmitted in batches via the UDP protocol; For low-value data, after aggregation and compression at edge nodes, it is transmitted in batches through a non-real-time channel during idle network periods.

3. The lightweight data transmission method for IoT devices based on edge computing according to claim 1, characterized in that, It also includes edge node state preservation mechanisms: The queue to be uploaded and the data list are persisted using non-volatile storage, and the persistent data is loaded to restore the breakpoint when the edge node restarts; Meanwhile, the queue to be uploaded adopts a circular buffer design. When the buffer is full, an alarm is triggered, and low-value data is discarded first to ensure stable system operation.

Images