Sparse code transmission method and system based on wavelet transform

Abstract

The invention discloses a sparse coding transmission method and system based on wavelet transform. The method comprises the steps of collecting a group of sensor samples for data processing, generating sparse coefficients by adopting lightweight Haar wavelet transform, screening important coefficients representing signal trends and abnormities (such as abrupt change) by utilizing a self-adaptive threshold value, constructing a data packet comprising a sparse mark, a threshold value, a low-frequency coefficient (signal stability) and a high-frequency coefficient (signal abrupt change and time index), and performing data processing on the data packet. And transmitting to an upper computer. And a receiving end can analyze the data packet with O (1) complexity to realize rapid detection of abnormity and realize signal reconstruction.

Description

Technical Field

[0001] The present invention relates to the field of Internet of Things sensor data processing, and specifically to a sparse coding transmission method and system based on wavelet transform, which is applicable to data compression and anomaly detection of resource-constrained embedded devices (such as Internet of Things sensors). Background Art

[0002] In the Internet of Things and industrial automation, sensors (such as temperature, vibration, electrocardiogram) generate analog signals with dynamic changes, which need to be processed and transmitted to the host computer in an embedded system with memory <1KB and processing time <0.1ms. The traditional per-sample transmission method sends one sample each time (for example, 2 bytes), resulting in a large amount of data (16 samples, 32 bytes), high bandwidth occupancy, and high energy consumption. It is suitable for high-bandwidth scenarios but not for low-speed networks (such as LoRa). Other compression methods such as discrete cosine transform (DCT) require complex matrix operations, with memory >10KB and time consumption >1ms; Huffman coding has a limited compression ratio (about 30–50%), and does not retain anomaly information in the time domain; JPEG2000 is based on wavelets but for images and has complex calculations. There is an urgent need for a lightweight and efficient method to achieve high compression ratio, fast anomaly detection, and low resource occupancy. Summary of the Invention

[0003] The present invention proposes a sparse coding transmission method and system based on wavelet transform. By utilizing the sparsity of signals in the wavelet domain, through lightweight Haar wavelet transform and adaptive threshold screening, compact data packets are generated, with a compression ratio of 50–90%, adapting to dynamic signal changes and quickly locating mutation points (such as temperature jumps).

[0004] The technical solution of the present invention includes the following core parts: 1. Data acquisition: Collect M samples (for example, M = 16, 100Hz, 0.16 seconds) to generate an analog signal (such as temperature). 2. Wavelet transform: Use Haar wavelets for 2-level decomposition to generate L low-frequency coefficients (L = 4, trends) and M - L high-frequency coefficients (M - L = 12, mutations), with a computational complexity of O(M). ● First-level decomposition: Pair every two of the M samples and calculate the average value (low frequency) and the difference value (high frequency): k low =(x i +x i+1 ) / 2 k high =(x i -x i+1 ) / 2 where k low and k high represent the low-frequency coefficient and the high-frequency sparsity respectively, and x iRepresents an analog value. ● Secondary decomposition: Re-pair and calculate the first-level low-frequency coefficients to generate lower-frequency and high-frequency coefficients. 3. Sparse coding: Select K high-frequency coefficients (K = 1–10) with an adaptive threshold (mean absolute value of high-frequency coefficients + 2σ), retain the values and indices, and retain all low-frequency coefficients. 4. Data packet construction: Include a sparsity flag (1 byte) + threshold (4 bytes) + L × 4-byte low-frequency coefficients + K × (4-byte value + 2-byte index), with a total size of 5 + 6K + 4L bytes. 5. Transmission: Send the data packet via Wi-Fi / LoRa. 6. Reconstruction and detection: The host computer parses the data packet, fills in the coefficients (set the rest to zero), reconstructs the signal through the inverse Haar transform, parses the high-frequency coefficient indices to locate mutations, with an error < 0.5 unit.

[0007] The system flow can be seen Figure 1 , including acquisition, transformation, coding, transmission, parsing, and reconstruction. ● The operation flow of the system modules is as follows: ■ Data acquisition module: Acquire sensor signals (such as temperature). ■ Wavelet transform module: Perform Haar wavelet decomposition to generate sparse coefficients. ■ Sparse coding module: Select coefficients to generate data packets. ■ Data transmission module: Send compressed data. ■ Signal reconstruction module: Reconstruct through inverse transform and detect mutations. ● The process of the system roles is as follows: ■ Lower computer: The data acquisition module acquires signals, the wavelet transform module generates coefficients, the sparse coding module selects and encodes, and the transmission module sends data packets. ■ Host computer: Receive data packets, the signal reconstruction module parses the coefficients, reconstructs the signal through inverse transform, and analyzes high-frequency coefficients to detect mutations.

[0008] The interaction logic of each module is as follows: ● The data acquisition module generates non-zero ΔT and transfers it to the sparsity calculation module. ● The sparsity calculation module determines N / M, triggers the update of T_base and pushes it onto the stack. ● The message generation module retrieves T_base from the stack head and constructs a message. ● The data reconstruction module parses the message and outputs the reconstructed data. ● The error control module verifies data and time errors.

[0009] The advantages of the present invention include: ● Compression ratio of 50–90%, memory < 100 bytes, time consumption < 0.1 ms. ● Anomaly detection <0.01ms, quickly locate mutations through high-frequency coefficient indexing. ● Adaptive threshold adapts to dynamic signals, retaining trends and mutations. ● Noise robust, filtering small coefficients, enhancing parsing reliability. ● Adapt to low-speed networks and embedded systems.

[0010] The present invention realizes data compression through sparse matrix design. The matrix only stores non-zero ΔT, the index is explicitly specified by Δt, and the parsing complexity is O(N). The stack mechanism ensures the reuse of T_base, reducing the overhead of recalculation. The method is applicable to scenarios such as Internet of Things devices and industrial sensors. Description of the Drawings

[0011] Figure 1 is a data processing flowchart;

[0012] Figure 2 Schematic diagram of sparse coefficients and packet structure, where the structure struct contains values and indexes;

[0013] Figure 3 Transmission and reconstruction timing diagram. Detailed Description of the Invention

[0014] The present invention will be further described in detail below in conjunction with the drawings and specific embodiments, but the embodiments of the present invention are not limited thereto.

[0015] Example of implementation: Temperature sensor, 100Hz, M = 16 (0.16 seconds), monitoring the ambient temperature (stable at 25°C, mutated to 50°C).

[0016] Lower computer processing: ● Data acquisition: Acquire 16 samples (32 bytes): ● Haar wavelet transform (2-level decomposition) ■ First-level decomposition (16 samples → 8 low-frequency + 8 high-frequency): Pair every two samples, and the calculation formula is as follows: k low =(x i +x i+1 ) / 2 k high =(x i -x i+1 ) / 2 where k low and k high represent the low-frequency coefficient and high-frequency sparsity respectively, and x i represents the analog value. ■ Secondary decomposition (8 low frequencies → 4 low frequencies + 4 high frequencies): Calculate the pairing of the first-level low frequencies: Low frequency 25.25 25.05 25.35 31.35 High frequency 0.05 0.05 0.05 -6.15 ● Total high-frequency coefficients (12): ● Meanings of high-frequency coefficients: ■ Small values (such as -0.1, 0.05): Indicate stable signal fluctuations or noise (for example, a small change of 25.2 → 25.4 °C). ■ Large values (such as -12.5, -6.15): Indicate signal mutations (for example, a drastic change of 25.0 → 50.0 °C). ■ Time index: The first-level high-frequency index 7 (-12.5) corresponds to samples [14, 15] (0.14–0.15 seconds); the second-level high-frequency index 3 (-6.15) corresponds to the first-level low frequency [6, 7] (samples [12–15]).

[0017] Sparse coding: ● Calculate the adaptive threshold: ■ Absolute values of high-frequency coefficients: 0.1×7, 12.5, 0.05×3, 6.15 ■ Mean: (0.1×7 + 12.5 + 0.05×3 + 6.15) / 12 = 19.5 / 12 ≈ 1.98 ■ Standard deviation: √[(7×(0.1 - 1.98)^2 + (12.5 - 1.98)^2 + 3×(0.05 - 1.98)^2 + (6.15 - 1.98)^2) / 12] ≈ 4.5 ■ Threshold: 1.98 + 2×4.5 = 10.98 ● Screening: Retain high-frequency coefficients with |coefficient| > 10.98, only K = 1 (-12.5, index 15), and retain all low frequencies (L = 4).

[0018] Data packet construction and transmission: ● Flag: K = 1 (1 byte) ● Threshold: 10.98 (4 bytes) ● Low-frequency coefficients: 25.25, 25.05, 25.35, 31.35 (4×4 = 16 bytes) ● High-frequency coefficients: (-12.5, index 15) (4 + 2 = 6 bytes) ● Total size: 1 + 4 + 16 + 6 = 27 bytes ● Transmission: Send via Wi-Fi / LoRa

[0019] Host computer processing: ● Parsing: Extract K = 1, threshold 10.98, low frequency (4), high frequency (-12.5, index 15). ● Mutation detection: Index 15 (0.15 s), value -12.5 (> 10.98), confirm mutation (50 °C), Time consumption <0.01 ms. ● Reconstruction: Fill 4 low frequencies + 1 high frequency (the remaining 11 high frequencies are set to zero), inverse Haar transform to restore 16 samples, error <0.5 °C.

[0020] Comparative advantages with traditional methods or other methods Based on the above embodiments, compared with traditional per-sample transmission and other compression methods (DCT, Huffman), the present invention has the following advantages: 1. Fast mutation detection: ● The present invention: The high-frequency coefficient (-12.5, index 15) directly indicates mutation (0.15 s, 50 °C), parsing O(1), <0.01 ms. ● Per-sample transmission: Compare 16 samples one by one (|x_15 - x_14| = 25 > threshold), O(n) = O(16), ~0.1 ms, noise needs to be filtered. ● DCT: The frequency-domain coefficient has no time index, needs to be scanned after inverse transformation, >0.1 ms. ● Huffman: No feature extraction, needs full-signal analysis, O(n). ● Advantage: Explicit index improves speed by 10 times, suitable for fire monitoring and electrocardiogram anomaly detection. 2. High-efficiency data compression: ● The present invention: 32 bytes → 27 bytes (16%, <20 bytes after optimization, >50%), single transmission. ● Per-sample transmission: 32 bytes, send 2 bytes 16 times, high bandwidth occupancy. ● DCT: Compression rate 30–50%, needs all frequency-domain coefficients (>32 bytes). ● Huffman: Compression rate 20–40% (~25 bytes), does not retain time features. ● Advantage: High compression rate, adapted to low-speed networks (LoRa), reduces energy consumption. 3. Accurate trend retention: ● The present invention: 4 low-frequency coefficients (25.25–31.35) capture the trend, error <0.5 °C. ● Per-sample transmission: Redundant fluctuations (25.2–25.4 °C), needs moving average (O(n)). ● DCT: High-frequency truncation distortion mutation. ● Huffman: No frequency decomposition, complex trend extraction. ● Advantage: Directly provides trends, reduces host computer calculation. 4. Noise robustness: ● This invention: Threshold (10.98) filters noise (-0.1), mutations (-12.5) are prominent. ● Per-sample transmission: Noise is directly transmitted, requires filtering (0.1ms delay). ● DCT: Noise is dispersed, truncation affects accuracy. ● Huffman: Compresses noise, reduces accuracy. ● Advantage: Enhances mutation reliability, suitable for noisy environments. 5. Low resource occupancy: ● This invention: Memory <100 bytes, time consumption <0.1ms, adapted to ESP32. ● Per-sample transmission: Memory <10 bytes, 16 times of sending increases energy consumption. ● DCT: Memory >10KB, time consumption >1ms. ● Huffman: Memory ~1KB, encoding ~0.5ms. ● Advantage: Lightweight, adapted to embedded systems with memory <1KB. 6. Dynamic signal adaptation: ● This invention: Adaptive threshold (mean + 2σ) adapts to stationary (K = 1) or mutated (K = 10). ● Per-sample transmission: Fixed sampling rate, redundant for stationary, missed sampling for mutations. ● DCT: Fixed frequency domain truncation, not suitable for time mutations. ● Huffman: No dynamic adjustment. ● Advantage: Flexibly responds to complex signals. 7. Multi-scale analysis: ● This invention: 2-level decomposition separates trends and high frequencies, supports selective analysis. ● Per-sample transmission: No frequency decomposition, requires additional algorithms. ● DCT: The frequency domain does not support time indexing. ● Huffman: No feature extraction. ● Advantage: Provides multi-scale features, suitable for fault diagnosis and electrocardiogram analysis.

[0021] In summary, this invention realizes the efficient compression and anomaly detection of Internet of Things sensor data through lightweight Haar wavelet transform and adaptive sparse coding, with a compression ratio of 50–90%, mutation detection <0.01ms, and memory <100 bytes, superior to traditional methods, and has significant industrial applicability.

Claims

1. A sparse coding transmission method and system based on wavelet transform, characterized in that It includes the following steps: ● A data acquisition module that acquires analog sensor signals and generates M samples; ● A wavelet transform module that performs wavelet transform on the sample data to generate multi-scale sparse coefficients, including low-frequency coefficients and high-frequency coefficients; ● A sparse coding module that filters important high-frequency coefficients according to an adaptive threshold, retains the coefficients whose values are higher than the threshold and their time indices, and retains all low-frequency coefficients; ● A data transmission module that sends data packets to the host computer. The data packets contain a sparsity flag, a threshold, low-frequency coefficients, and high-frequency coefficients (value + index); ● A signal reconstruction module that the host computer parses the data packets, fills in the coefficients, sets the rest to zero, performs inverse wavelet transform to reconstruct the signal and detect anomalies.

2. The method according to claim 1, wherein The wavelet transform uses a lightweight Haar wavelet basis and generates coefficients by successively calculating the mean and difference of sample pairs.

3. The method according to claim 1, wherein The adaptive threshold is calculated based on the mean of the absolute values of the high-frequency coefficients plus k times the standard deviation, where k is a preset constant and dynamically adapts to signal changes.

4. The method according to claim 1, characterized in that The size of the data packet is 5 + 6K + 4L bytes, where K is the number of high-frequency sparse coefficients, L is the number of low-frequency coefficients, and it contains 1 byte of sparsity flag, 4 bytes of threshold, L 4-byte low-frequency coefficients, and K (4-byte value + 2-byte index).

5. The method according to claim 1, characterized in that, The high-frequency coefficients represent the rapid changes of the signal. Small values correspond to stable fluctuations or noise, and large values correspond to mutations. The time index locates the mutation points.

6. The method according to claim 1, characterized in that, The host computer detects signal mutation points by parsing the indices and values of the high-frequency sparse coefficients.

Images