Device data processing method and apparatus

Abstract

The application provides a device data processing method and device. The method comprises the following steps: collecting device data based on a preset sampling frequency; determining the fluctuation degree of the device data based on a plurality of time sliding windows; adjusting the sampling frequency based on the fluctuation degree of the device data; determining the compression mode of the device data based on the data type, the fluctuation degree and the importance degree of the device data; compressing the device data based on the compression mode; determining the uploading mode and the uploading frequency of the device data based on the data type, the fluctuation degree and the importance degree of the device data; and uploading the device data based on the uploading mode and the uploading frequency. In the method, the data change trend can be perceived in real time, the sampling frequency can be dynamically adjusted according to the data fluctuation, the collected data can be classified and compressed and stored locally, and intelligent uploading can be performed in combination with a priority label management strategy.

Description

Technical Field

[0001] This invention relates to the field of smart home technology, and in particular to a device data processing method and apparatus. Background Technology

[0002] Kitchen appliances generate cooking curve data during the cooking process. For example, at the start of cooking, the device periodically reports its current status every 2 seconds, including detected temperature, humidity, and cooking mode. When the device starts from room temperature and enters the preheating phase (usually within 5 minutes), the temperature and humidity values ​​undergo a period of rapid change. However, once the device reaches the preset temperature, even if the cooking time is set to 45 minutes and the device continues to run, the temperature changes will no longer be significant. Even so, the device continues to report data at a fixed frequency (every 2 seconds) until cooking is finished. This practice not only consumes device performance and impacts the device's more critical communication data, but also leads to unnecessary waste of cloud resources and storage.

[0003] In the aforementioned scenarios, sensors continuously generate large amounts of time-series data in environments such as smart devices, industrial IoT, and edge computing nodes. This data is often sampled and uploaded at a fixed frequency, leading to the following increasingly prominent problems: 1. Severe network bandwidth burden, especially in systems with a large number of edge node devices; 2. Limited storage space on local devices, making it impossible to effectively preserve historical data; 3. Increased pressure on cloud data processing, especially with the large-scale uploading of highly redundant, low-value data; 4. Real-time data analysis systems are disrupted by invalid data, reducing decision-making accuracy. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a device data processing method and apparatus to effectively reduce data redundancy, alleviate the burden on devices and the cloud, and improve the overall operating efficiency of the system while ensuring the quality of key data.

[0005] In a first aspect, embodiments of the present invention provide a device data processing method, the method comprising: collecting device data based on a preset sampling frequency; determining the fluctuation level of the device data based on multiple time sliding windows; adjusting the sampling frequency based on the fluctuation level of the device data; determining a compression method for the device data based on the data type, fluctuation level, and importance of the device data; compressing the device data based on the compression method; determining an upload method and upload frequency for the device data based on the data type, fluctuation level, and importance of the device data; and uploading the device data based on the upload method and upload frequency.

[0006] In an optional embodiment of this application, the step of determining the fluctuation level of device data based on multiple time sliding windows includes: using multiple preset time lengths as the lengths of multiple time sliding windows; determining the average fluctuation level of the multiple time sliding windows based on device data; and performing a weighted calculation based on the average fluctuation level of the multiple time sliding windows to obtain the fluctuation level of device data.

[0007] In an optional embodiment of this application, the step of determining the average fluctuation of multiple time sliding windows based on device data includes: obtaining the currently collected raw value; determining the target value of the sampling points before the raw value of the multiple time sliding windows based on the length of the multiple time sliding windows; and determining the average fluctuation of the multiple time sliding windows based on the raw value and the target value of the multiple time sliding windows.

[0008] In an optional embodiment of this application, the step of adjusting the sampling frequency based on the fluctuation level of device data includes: if the fluctuation level of device data is continuously less than a preset first threshold and the duration is greater than a preset duration threshold, the sampling frequency is reduced; if the fluctuation level of device data is continuously greater than a preset second threshold and the number of consecutive occurrences is greater than a preset number of occurrences threshold, the sampling frequency is increased; wherein, the second threshold is greater than the first threshold.

[0009] In an optional embodiment of this application, after the above-described step of reducing the sampling frequency, the method further includes: if the reduced sampling frequency is less than a preset minimum active frequency, setting the sampling frequency to the minimum active frequency.

[0010] In optional embodiments of this application, the above data types include: business data, alarm data, and system anomaly data; the compression methods include: differential coding, run-length encoding, variable precision compression, and data compression methods based on dictionaries and pattern matching.

[0011] In optional embodiments of this application, the steps of determining the upload method and upload frequency of device data based on the data type, fluctuation level, and importance of device data include: determining the priority label of device data based on the data type, fluctuation level, and importance of device data; and determining the upload method and upload frequency of device data based on the priority label.

[0012] In optional embodiments of this application, the above-mentioned upload methods include: real-time or near real-time upload, batch upload, and upload when there is a network connection; the upload frequency includes: high-frequency upload, medium-frequency upload, low-frequency upload, and delayed upload.

[0013] In optional embodiments of this application, the above method further includes: if the device experiences a network failure or cache overflow, determining the device data summary data and uploading the compressed summary data; after the device's network is restored or the cache is restored, re-uploading the missing device data so that the device data uploader can complete and compare the device data using tags and timestamps.

[0014] Secondly, embodiments of the present invention also provide a device data processing apparatus, comprising: a dynamic sampling module, configured to collect device data based on a preset sampling frequency, determine the fluctuation level of the device data based on multiple time sliding windows, and adjust the sampling frequency based on the fluctuation level of the device data; a differentiated compression module, configured to determine the compression method of the device data based on the data type, fluctuation level, and importance of the device data, and compress the device data based on the compression method; and a tag-based priority management module, configured to determine the upload method and upload frequency of the device data based on the data type, fluctuation level, and importance of the device data, and upload the device data based on the upload method and upload frequency.

[0015] The embodiments of the present invention bring the following beneficial effects: This invention provides a device data processing method and apparatus. The method involves collecting device data based on a preset sampling frequency, determining the fluctuation level of the device data based on multiple time sliding windows, and adjusting the sampling frequency based on the fluctuation level. It also involves determining a compression method for the device data based on its data type, fluctuation level, and importance, and compressing the device data accordingly. Finally, it involves determining an upload method and upload frequency for the device data based on its data type, fluctuation level, and importance, and uploading the device data based on these parameters. This method allows for real-time sensing of data change trends, dynamic adjustment of the sampling frequency based on data fluctuations, hierarchical compression and local storage of the collected data, and intelligent uploading combined with a priority tag management strategy.

[0016] Other features and advantages of this disclosure will be set forth in the following description, or some features and advantages may be inferred from the description or determined without doubt, or may be learned by practicing the techniques described above.

[0017] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0019] Figure 1 A flowchart of a device data processing method provided in an embodiment of the present invention; Figure 2 A flowchart of another device data processing method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a device data processing apparatus provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Currently, in environments such as smart devices, industrial IoT, and edge computing nodes, sensors continuously generate large amounts of time-series data. This data is often sampled and uploaded at a fixed frequency, leading to the following increasingly prominent problems: 1. Severe network bandwidth burden, especially in systems with a large number of edge node devices; 2. Limited storage space on local devices, making it impossible to effectively preserve historical data; 3. Increased pressure on cloud data processing, especially with the large-scale uploading of highly redundant, low-value data; 4. Real-time data analysis systems are disrupted by invalid data, reducing decision-making accuracy.

[0022] Based on this, the present invention provides a device data processing method and apparatus, specifically a device data processing method for dynamic frequency reduction acquisition and data compression storage, which enhances the efficiency, storage capacity and intelligent upload scheduling function of edge devices in processing massive amounts of operational data under limited computing resources and bandwidth conditions, and ensures that key data can be uploaded to the cloud in a priority, stable and efficient manner in low-bandwidth network environments.

[0023] To facilitate understanding of this embodiment, a device data processing method disclosed in this embodiment of the invention will first be described in detail.

[0024] Example 1: This invention provides a device data processing method, see [link to relevant documentation]. Figure 1 The flowchart shown illustrates a device data processing method, which includes the following steps: Step S102: Collect device data based on a preset sampling frequency, determine the fluctuation level of the device data based on multiple time sliding windows, and adjust the sampling frequency based on the fluctuation level of the device data.

[0025] In this embodiment, the device data can be the raw data collected from various sensors or subsystems on the device, without any processing, compression, or filtering. Taking a steam oven as an example, the device data can be shown in Table 1: Table 1

[0026] This embodiment provides a dynamic sampling mechanism. Based on the device data change rate, multiple time sliding window fluctuation detection and trend perception algorithms, the stability or jitter of the sampling interval of multiple time sliding windows is judged in real time as the degree of fluctuation of device data. The sampling frequency of device data is dynamically adjusted based on the degree of fluctuation, so as to achieve sensitive capture of abnormal changes and reduced frequency acquisition of normal data.

[0027] Step S104: Determine the compression method of the device data based on the data type, fluctuation level, and importance of the device data, and compress the device data based on the compression method.

[0028] This embodiment provides a differentiated compression strategy, which determines different compression methods based on the data type, fluctuation level, and importance of the device data. It can perform multi-layer compression encoding on the collected device data, including fixed-point compression, residual prediction compression, and variable-precision quantization storage, thereby improving the compression ratio while retaining reconstruction capabilities.

[0029] Step S106: Determine the upload method and upload frequency of the device data based on the data type, fluctuation level, and importance of the device data, and upload the device data based on the upload method and upload frequency.

[0030] This embodiment provides a tag-based priority management method, which can tag device data with priority tags (such as critical events, high-risk indicators, regular fluctuations, etc.), and establish a structured local cache directory with priority tags as indexes, in conjunction with a lightweight index database to manage storage and access.

[0031] This invention provides a device data processing method. The method involves collecting device data based on a preset sampling frequency, determining the fluctuation level of the device data using multiple time sliding windows, and adjusting the sampling frequency based on the fluctuation level. It also involves determining a compression method for the device data based on its data type, fluctuation level, and importance, and compressing the device data accordingly. Finally, it determines an upload method and upload frequency based on the same data type, fluctuation level, and importance, and uploads the device data based on these parameters. This method allows for real-time sensing of data change trends, dynamic adjustment of the sampling frequency based on data fluctuations, hierarchical compression and local storage of the collected data, and intelligent uploading combined with a priority tagging management strategy.

[0032] Example 2: This embodiment provides another device data processing method, which is implemented based on the above embodiment. See [link to relevant documentation]. Figure 2 The flowchart shown represents another device data processing method, which includes the following steps: Step S202: Collect device data based on a preset sampling frequency, determine the fluctuation level of the device data based on multiple time sliding windows, and adjust the sampling frequency based on the fluctuation level of the device data.

[0033] In some embodiments, multiple preset time lengths can be used as the lengths of multiple time sliding windows; the average fluctuation of the multiple time sliding windows is determined based on device data; and a weighted calculation is performed based on the average fluctuation of the multiple time sliding windows to obtain the fluctuation of the device data.

[0034] This embodiment supports different sampling accuracies, such as high, medium, and low. A fluctuation detector can monitor the short-term, medium-term, and long-term changes in device data in real time; the scheduling controller dynamically sets the sampling frequency based on the detection results. For example, the fluctuation of sensor data collected by the device has phased characteristics: short-term fluctuations may be drastic (e.g., during the preheating phase of the all-in-one machine), medium-term fluctuations may be relatively stable (e.g., after the all-in-one machine reaches the set temperature), and long-term fluctuations may show a slow trend (e.g., as the all-in-one machine nears the end of its operation).

[0035] In order to accurately perceive the trend of data changes, this embodiment can calculate the average fluctuation degree through a multi-scale sliding window (short / mid / long), and then calculate a comprehensive fluctuation score in a weighted manner as the fluctuation degree of the device data, dynamically adjusting the sampling frequency to save resources without losing important information.

[0036] In some embodiments, the currently acquired raw value can be obtained, and the target value of the sampling points before the raw value of the multiple time sliding windows can be determined based on the length of the multiple time sliding windows respectively; the average fluctuation degree of the multiple time sliding windows can be determined based on the raw value and the target value of the multiple time sliding windows respectively.

[0037] For example, this embodiment can set three time sliding windows: high, medium, and low, and calculate the average fluctuation of these three time sliding windows: ; ; ; in, This represents the average fluctuation level of a low-level time sliding window. This represents the average fluctuation level of a medium-range time sliding window. This represents the average fluctuation level of a high-end time sliding window.

[0038] in, This is the currently collected raw value; The target value before the i-th sampling point; , , These represent the sizes of the time sliding windows for low, medium, and high time settings (adjustable), for example, , , ; The result represents the average degree of fluctuation per unit time within that window.

[0039] After obtaining the average volatility level, this embodiment can further calculate the weighted volatility score: Fluctuation Score = α × +β× +γ× ; The Fluctuation Score represents the final weighted volatility; a higher value indicates greater volatility, and vice versa. α, β, and γ are different weighting coefficients. α represents the sensitivity to short-term sharp fluctuations, responding to operational abrupt changes, and can be 0.6; β represents the weight of medium-term trend changes, and can be 0.3; γ represents the ability to regulate long-term stability, and can be 0.1.

[0040] This embodiment allows for flexible adjustment of these three weights to adapt to different equipment types or business needs (such as steam ovens and ovens).

[0041] In some embodiments, if the fluctuation level of the device data is continuously less than a preset first threshold and the duration is greater than a preset duration threshold, the sampling frequency is reduced; if the fluctuation level of the device data is continuously greater than a preset second threshold and the number of consecutive occurrences is greater than a preset number threshold, the sampling frequency is increased; wherein, the second threshold is greater than the first threshold.

[0042] The sampling frequency adjustment rule in this embodiment can be: if Fluctuation Score <T1: Reduce sampling frequency (frequency reduction processing) elif Fluctuation Score>T2: Increase the sampling frequency (frequency upsampling) else: Maintain the current frequency.

[0043] The first threshold T1 and the second threshold T2 are the upper and lower fluctuation thresholds set by the system, which have a hysteresis and rebound mechanism (anti-shake). The sampling frequency is increased only when the Fluctuation Score exceeds the second threshold T2 multiple times consecutively. The sampling frequency is decreased only when the Fluctuation Score drops below the first threshold T1 and remains below it for a period of time.

[0044] The hysteresis and rebound mechanism in this embodiment can be as follows: the current sampling frequency is consistently higher than T2, and the frequency is increased only after N consecutive times; the current sampling frequency is occasionally lower than T1, and no change is made; the current sampling frequency is consistently lower than T1, and the frequency is reduced after N consecutive times.

[0045] In some embodiments, if the reduced sampling frequency is less than the preset minimum active frequency, the sampling frequency is set to the minimum active frequency.

[0046] This embodiment can perform frequency reduction or increase processing in the following way: the scheduler switches between multiple preset frequency levels according to the scoring results, such as: high frequency: 10Hz, medium frequency: 1Hz, low frequency: 0.1Hz, and supports custom interpolation or exponential adjustment methods to achieve continuous adjustment, while setting a minimum active frequency to prevent complete stoppage of sampling.

[0047] This invention provides a dynamic sampling mechanism based on fluctuation sensing. It assesses data volatility through three sliding windows (short-term, medium-term, and long-term) and dynamically adjusts the sampling frequency. This can significantly reduce the collection of invalid or redundant data and lower device power consumption.

[0048] Step S204: Determine the compression method for the device data based on its data type, volatility, and importance, and compress the device data based on the compression method.

[0049] In some embodiments, the data types include: business data, alarm data, and system anomaly data; the compression methods include: differential coding, run-length encoding, variable precision compression, and data compression methods based on dictionaries and pattern matching.

[0050] In this embodiment, based on characteristics such as data type (business data, alarm data, system anomaly data), volatility (obtained by a dynamic sampling mechanism), and importance (low, medium, high), adjustable precision compression, compression algorithm selection, and hierarchical compression structure management can be implemented to improve compression efficiency and the fidelity of key data.

[0051] This embodiment can extract the following tags based on data characteristics: Data type: temperature (heating cavity, door, probe), humidity (cavity humidity), electrical data (voltage, current, heating power), image (cavity camera), user operation status, environmental parameters, etc.; rate of change (output from the dynamic sampling stage); importance level: key indicators, general indicators, redundant indicators.

[0052] The compression strategy in this embodiment can be as follows: if the label combination is high-frequency change + key indicator, the compression strategy is decimal point retention + Delta encoding (differential encoding); if the label combination is low-frequency change + general indicator, the compression strategy is variable precision encoding + Run-Length encoding (RLE); if the label combination is low-frequency change + redundant indicator, the compression strategy is discarding part of the precision + fixed-ratio downsampling; if the label combination is a stable data block, the compression strategy is direct hash identification + deduplication compression.

[0053] The effects of various compression strategies are as follows: 1. Delta encoding (differential encoding): Records the change (difference) between adjacent data points, rather than the original value itself. Example: Original data [100,102,105] → Difference result [100,+2,+3].

[0054] 2. Run-Length Encoding (RLE): Data that is repeated consecutively is recorded only once and the number of times it is repeated. Example: [50,50,50,20] → [(50,3),(20,1)].

[0055] 3. Floating-point Quantization: Truncates floating-point data according to a set precision (e.g., retaining 1 or 2 decimal places). Example: Original temperature 76.2487°C → retain 76.2°C (precision set to 1 decimal place).

[0056] 4. General compression algorithms such as Snappy: General data compression algorithms based on dictionaries and pattern matching, which can achieve fast compression / decompression without being specific to a certain data structure, and are suitable for embedded scenarios.

[0057] This invention provides a differentiated compression strategy that selects compression methods such as Delta encoding, variable precision quantization, RLE, and Snappy based on data characteristics to achieve customized compression. This can improve the compression ratio, save local storage space, and support flexible control of data precision to meet different upload / analysis needs.

[0058] Step S206: Determine the upload method and upload frequency of the device data based on the data type, fluctuation level, and importance of the device data, and upload the device data based on the upload method and upload frequency.

[0059] In some embodiments, priority tags for device data can be determined based on the data type, volatility, and importance of the device data; the upload method and upload frequency of the device data can be determined based on the priority tags.

[0060] In this embodiment, an upload system can be designed that combines local structured caching, intelligent hierarchical upload mechanism and tag-based priority scheduling, supporting "time-sharing, hierarchical, and batch" transmission strategies, and dynamically adapting to the network environment and the urgency of the data.

[0061] For example, the data cache directory is divided into: / data / buffer / high-priority / , / data / buffer / normal / , and / data / buffer / low / ; each data packet carries the following metadata: { "tag": "high-priority", "timestamp": "2025-06-10T12:34:56", "hash": "xx", "compression": "delta+quantized", "size": 128 } In some embodiments, the upload method includes: real-time or near real-time upload, batch upload, and upload when there is a network connection; the upload frequency includes: high-frequency upload, medium-frequency upload, low-frequency upload, and delayed upload.

[0062] For example, if the tag level is high-priority, the upload method is real-time or near real-time upload, and the upload frequency is high; if the tag level is normal, the upload method is batch upload, and the upload frequency is medium; if the tag level is low-priority, the upload method is upload only when there is internet access, and the upload frequency is low / delayed upload.

[0063] Step S208: If the device experiences a network failure or cache overflow, determine the device data summary data and upload the compressed summary data; after the device's network or cache is restored, re-upload the missing device data so that the device data uploader can complete and compare the device data using tags and timestamps.

[0064] This embodiment can also provide an abnormal data compensation mechanism, which combines the network status of the device, time window judgment, tag priority scheduling and confirmation retransmission mechanism to prioritize the upload of high-value data and supplement the upload of low-priority data when bandwidth is sufficient, and supports breakpoint resume, batch backhaul and manual triggering backhaul.

[0065] If the device loses network access or the cache overflows, the summary data is retained, and a compressed summary is uploaded. Once the network is restored, the missing data is retransmitted, and the cloud uses tags and timestamps to complete and compare the data.

[0066] This invention provides an intelligent hierarchical storage and upload mechanism that enables on-demand uploading and latency tolerance, improving network adaptability and ensuring the real-time transmission and analysis value of high-priority data.

[0067] In summary, this invention provides a device-side data processing method that combines a dynamic sampling mechanism with a differentiated compression strategy. By sensing data change trends in real time and dynamically adjusting the sampling frequency based on data volatility, the method performs hierarchical compression and local storage on the collected data. Combined with a priority tag management strategy and an intelligent upload mechanism, it achieves the following technical breakthroughs: dynamic sampling frequency reduction is achieved by combining multi-window differential and volatility sensing algorithms; a "differentiated compression" strategy is proposed to support variable-precision storage and data return; a tag indexing mechanism is used to support priority classification management and on-demand reconstruction; and a local structured storage + intelligent hierarchical upload mechanism is constructed to improve the intelligence and flexibility of the upload strategy.

[0068] The method provided in the embodiments of the present invention can solve the following technical problems: 1. The problem of data redundancy and resource waste caused by fixed sampling frequency at the device end: This invention combines a multi-window difference algorithm with a volatility sensing model to achieve real-time identification of data change trends and dynamically adjust the sampling frequency. This reduces sampling density when data changes are stable, minimizes redundant data acquisition, and significantly saves acquisition and processing resources.

[0069] 2. The problem of low compression efficiency and loss of important data information coexisting under a unified compression strategy: This invention proposes a "differentiated compression" strategy, which supports hierarchical compression and storage based on the importance and accuracy requirements of the data, achieving high fidelity for high-value data and high compression for low-value data, thereby improving the overall compression effect and storage utilization.

[0070] 3. The lack of fine-grained management and priority scheduling during data upload and storage: This invention constructs a tag index system based on multi-dimensional characteristics such as data volatility, anomaly level, and time sensitivity to achieve priority classification management and on-demand data reconstruction, ensuring real-time uploading and timely response of critical data. Simultaneously, it dynamically determines the upload time and data granularity based on network conditions, data level, and cloud processing capabilities to maximize bandwidth utilization efficiency.

[0071] Example 3: Corresponding to the above method embodiments, this invention provides a device data processing apparatus, see [link to previous document]. Figure 3 The diagram shows a structural schematic of a device data processing apparatus, which includes: The dynamic sampling module 31 is used to collect device data based on a preset sampling frequency, determine the fluctuation level of the device data based on multiple time sliding windows, and adjust the sampling frequency based on the fluctuation level of the device data. Differentiated compression module 32 is used to determine the compression method of device data based on the data type, fluctuation level and importance of device data, and compress device data based on the compression method; The tagging priority management module 33 is used to determine the upload method and upload frequency of device data based on the data type, fluctuation level and importance of the device data, and upload device data based on the upload method and upload frequency.

[0072] This invention provides a device data processing apparatus that collects device data based on a preset sampling frequency, determines the fluctuation level of the device data based on multiple time sliding windows, and adjusts the sampling frequency based on the fluctuation level of the device data; determines the compression method of the device data based on the data type, fluctuation level, and importance, and compresses the device data based on the compression method; and determines the upload method and upload frequency of the device data based on the data type, fluctuation level, and importance, and uploads the device data based on the upload method and upload frequency. In this method, the data change trend can be perceived in real time, the sampling frequency can be dynamically adjusted according to the data fluctuation, the collected data is subjected to hierarchical compression and local storage, and intelligent upload is performed in conjunction with a priority tag management strategy.

[0073] The aforementioned dynamic sampling module is used to use multiple preset time lengths as the lengths of multiple time sliding windows; determine the average fluctuation level of the multiple time sliding windows based on device data; and perform a weighted calculation based on the average fluctuation level of the multiple time sliding windows to obtain the fluctuation level of the device data.

[0074] The aforementioned dynamic sampling module is used to acquire the currently collected raw value, determine the target value of the sampling points before the raw value of each of the multiple time sliding windows based on the length of the multiple time sliding windows, and determine the average fluctuation degree of the multiple time sliding windows based on the raw value and the target value of each of the multiple time sliding windows.

[0075] The aforementioned dynamic sampling module is used to reduce the sampling frequency if the fluctuation level of the device data is continuously less than a preset first threshold and the duration is greater than a preset duration threshold; and to increase the sampling frequency if the fluctuation level of the device data is continuously greater than a preset second threshold and the number of consecutive occurrences is greater than a preset number threshold; wherein the second threshold is greater than the first threshold.

[0076] The aforementioned dynamic sampling module is also used to set the sampling frequency to the minimum active frequency if the reduced sampling frequency is less than the preset minimum active frequency.

[0077] The data types mentioned above include: business data, alarm data, and system anomaly data; the compression methods mentioned above include: differential coding, run-length encoding, variable precision compression, and data compression methods based on dictionaries and pattern matching.

[0078] The aforementioned tagging priority management module is used to determine the priority tags of device data based on the data type, fluctuation level, and importance of the device data; and to determine the upload method and upload frequency of device data based on the priority tags.

[0079] The above upload methods include: real-time or near real-time upload, batch upload, and upload when there is a network connection; the above upload frequencies include: high-frequency upload, medium-frequency upload, low-frequency upload, and delayed upload.

[0080] The aforementioned device further includes: a device data retransmission module, used to determine the summary data of the device data and upload the compressed summary data if the device experiences a network failure or cache overflow; and to re-upload the missing data of the device data after the device's network or cache is restored, so that the device data uploader and receiver can complete and compare the device data through tags and timestamps.

[0081] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device data processing apparatus described above can be referred to the corresponding process in the embodiments of the aforementioned device data processing method, and will not be repeated here.

[0082] Example 4: This invention also provides an electronic device for running the above-described device data processing method; see [link to previous document]. Figure 4 The diagram shows the structure of an electronic device, which includes a memory 100 and a processor 101. The memory 100 is used to store one or more computer instructions, which are executed by the processor 101 to implement the above-mentioned device data processing method.

[0083] Furthermore, Figure 4 The electronic device shown also includes a bus 102 and a communication interface 103, with the processor 101, the communication interface 103 and the memory 100 connected via the bus 102.

[0084] The memory 100 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 103 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 102 may be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 4 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0085] Processor 101 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of processor 101 or by instructions in software form. Processor 101 can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this invention can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software module can reside in a readily available storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory 100, and processor 101 reads information from memory 100 and, in conjunction with its hardware, completes the steps of the method described in the foregoing embodiments.

[0086] This invention also provides a computer-readable storage medium storing computer-executable instructions. When these computer-executable instructions are called and executed by a processor, they cause the processor to implement the aforementioned device data processing method. For specific implementation details, please refer to the method embodiments, which will not be repeated here.

[0087] The computer program product of the device data processing method and apparatus provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0088] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and / or device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0089] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0090] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0091] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0092] Finally, it should be noted that the above embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A device data processing method, characterized in that, The method includes: Data from the device is collected based on a preset sampling frequency, the fluctuation level of the device data is determined based on multiple time sliding windows, and the sampling frequency is adjusted based on the fluctuation level of the device data. The compression method for the device data is determined based on the data type, volatility, and importance of the device data, and the device data is compressed based on the compression method. The upload method and upload frequency of the device data are determined based on the data type, fluctuation level, and importance of the device data, and the device data is uploaded based on the upload method and upload frequency.

2. The method according to claim 1, characterized in that, The step of determining the degree of fluctuation of the device data based on multiple time sliding windows includes: Use multiple preset time lengths as the lengths of multiple time sliding windows; The average fluctuation of the multiple time sliding windows is determined based on the device data. The fluctuation level of the device data is obtained by weighting the average fluctuation level of multiple time sliding windows.

3. The method according to claim 2, characterized in that, The step of determining the average fluctuation of multiple time sliding windows based on the device data includes: Obtain the currently acquired raw value, and determine the target value of the sampling point before the raw value of the multiple time sliding windows based on the length of the multiple time sliding windows respectively; The average fluctuation of the multiple time sliding windows is determined based on the original value and the target value of the multiple time sliding windows.

4. The method according to claim 1, characterized in that, The step of adjusting the sampling frequency based on the fluctuation level of the device data includes: If the fluctuation of the device data is consistently less than a preset first threshold and the duration is greater than a preset duration threshold, the sampling frequency is reduced. If the fluctuation of the device data is continuously greater than a preset second threshold, and the number of consecutive occurrences is greater than a preset number threshold, the sampling frequency is increased; wherein, the second threshold is greater than the first threshold.

5. The method according to claim 4, characterized in that, After the step of reducing the sampling frequency, the method further includes: If the reduced sampling frequency is less than the preset minimum active frequency, the sampling frequency is set to the minimum active frequency.

6. The method according to claim 1, characterized in that, The data types include: business data, alarm data, and system anomaly data; The compression methods include: differential coding, run-length coding, variable precision compression, and data compression methods based on dictionaries and pattern matching.

7. The method according to claim 1, characterized in that, The steps for determining the upload method and upload frequency of the device data based on its data type, volatility, and importance include: The priority label of the device data is determined based on the data type, volatility, and importance of the device data; The upload method and upload frequency of the device data are determined based on the priority label.

8. The method according to claim 1, characterized in that, The upload methods include: real-time or near-real-time upload, batch upload, and upload only when there is internet access. The upload frequencies include: high-frequency upload, medium-frequency upload, low-frequency upload, and delayed upload.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: If the device experiences a network failure or cache overflow, determine the summary data of the device data and upload the compressed summary data. After the network or cache of the device is restored, the missing data of the device data is re-uploaded so that the recipient of the device data can complete and compare the device data using tags and timestamps.

10. A device data processing apparatus, characterized in that, The device includes: A dynamic sampling module is used to collect device data based on a preset sampling frequency, determine the fluctuation level of the device data based on multiple time sliding windows, and adjust the sampling frequency based on the fluctuation level of the device data. A differentiated compression module is used to determine the compression method of the device data based on the data type, volatility, and importance of the device data, and to compress the device data based on the compression method. The tagging priority management module is used to determine the upload method and upload frequency of the device data based on the data type, fluctuation level and importance of the device data, and upload the device data based on the upload method and upload frequency.

Images