A markov decision-based internet of things low-power transmission system and method

Abstract

The application provides a Markov decision-based Internet of Things low-power transmission system and method, relates to the field of data transmission, and the method comprises the following steps: collecting three-dimensional state data of a terminal; outputting an optimal action through an MDP decision module, performing data collection, compression, transmission and hibernation, and updating a Q table according to a transmission result; a gateway receives data sent by the terminal and forwards the data to the cloud, a user monitors a state and dynamically adjusts a strategy parameter through a cloud GUI, and the gateway issues a new strategy to the terminal for execution. The technical scheme of the application realizes the cooperation of hardware hibernation and software compression with low power consumption, the endurance performance is greatly improved, a three-layer architecture design is adopted, and the terminal autonomous decision and cloud regulation and control are considered.

Description

Technical Field

[0001] This application relates to the field of data transmission, and in particular to a low-power transmission system and method for the Internet of Things based on Markov decision-making. Background Technology

[0002] Existing IoT terminals generally adopt a technical solution of "fixed periodic data collection + fixed parameter transmission". The core measures are as follows: the terminal (such as the ESP32 series) is equipped with sensors (temperature, humidity, pressure, etc.) and communication modules (WiFi, LoRa, etc.). It collects data at preset fixed time intervals (such as 30 seconds / 1 minute), transmits the data to the gateway using a fixed compression ratio (such as no compression) and a fixed communication protocol (such as LoRa default parameters), and then the gateway aggregates and uploads the data to the cloud.

[0003] A typical application of this solution is in a traditional smart agriculture temperature and humidity monitoring system: the terminal collects temperature and humidity data every 30 seconds without data compression, and transmits it to the gateway via a LoRa module with a fixed spreading factor (SF=12). The gateway then uploads the data to the cloud platform for display via WiFi.

[0004] The above solution has significant drawbacks: excessive power consumption and poor battery life due to fixed high-frequency acquisition and transmission—with a 2000mAh lithium battery, the traditional fixed-cycle solution only lasts about 7 days, requiring frequent battery replacements and resulting in high maintenance costs; poor adaptability, unable to adapt to dynamic environmental changes—when the gateway moves and the signal weakens, fixed-parameter transmission will experience a packet loss rate of over 50%; when the data is in a stable state, high-frequency transmission causes data redundancy; insufficient reliability, lacking a dynamic adjustment mechanism—in extreme scenarios (such as when the terminal battery is below 20%), high-frequency transmission is maintained, leading to rapid power failure and loss of critical data. Summary of the Invention

[0005] The purpose of this invention is to address the problems of high power consumption, poor adaptability, and low reliability caused by the fixed parameter transmission strategy used in existing IoT terminals, and to provide a low-power transmission system and method for IoT based on Markov decision-making.

[0006] The above-mentioned objective of this application is achieved through the following technical solution: The terminal layer, gateway layer, and cloud layer are connected in sequence, with the terminal layer and cloud layer being connected. The terminal layer includes: a multi-dimensional state perception module, an MDP dynamic strategy decision-making module, a deep low-power control module, and an intelligent data processing module. A multi-dimensional status awareness module is used to collect the terminal's power status, communication quality, and data priority in real time. The MDP dynamic strategy decision module is used to select the optimal transmission action based on the output of the multi-dimensional state perception module and the lightweight MDP algorithm. A deep low-power control module is used to control the terminal to enter a deep sleep mode based on the output of the decision module. The intelligent data processing module is used to compress the collected data based on the decision results; The gateway layer includes: a dual communication protocol adaptation module, a data relay and caching module, and a policy distribution module; A dual communication protocol adapter module is used to enable LoRa communication between the terminal layer and the gateway layer and WiFi communication between the gateway layer and the cloud layer. The data relay and caching module is used to receive terminal data and cache it to local storage; The policy delivery module is used to receive policy parameters delivered by the cloud layer and broadcast them to the terminal. The cloud layer includes: a strategy parameter configuration module, a running status monitoring module, and a data storage and analysis module; The strategy parameter configuration module is used to configure and distribute MDP strategy parameters; The operation status monitoring module is used to monitor the terminal's operation status and issue abnormal alarms; The data storage and analysis module is used to store historical data and perform analysis.

[0007] Optionally, in the multi-dimensional state perception module: The battery status is divided into five levels based on the battery voltage; The communication quality is divided into four levels based on RSSI signal strength: Excellent, Good, Average, and Poor. The data priority is divided into four levels based on the fluctuation range of temperature and humidity: over-threshold warning, high fluctuation, stable and low change.

[0008] Optionally, the MDP dynamic strategy decision module specifically includes: By optimizing the lightweight MDP algorithm and the forgetting factor, a closed-loop decision-making model for state-action rewards is constructed, namely the MDP dynamic policy decision module. The state space of the closed-loop decision model consists of a combination of three-dimensional states: power status, communication quality, and data priority, resulting in a total of 80 discrete states. The action space of the closed-loop decision model consists of a two-dimensional combination of reporting period and compression ratio. The reporting period includes four levels: 10 seconds, 30 seconds, 120 seconds, and 900 seconds. The compression ratio includes three levels: 0%, 30%, and 60%, for a total of 12 actions. The reward function R of the closed-loop decision model is composed of a weighted sum of power consumption reward R1 and data integrity reward R2, with the specific formula as: R = 0.6R1 + 0.4R2, where R1 = (baseline power consumption - actual power consumption) / baseline power consumption * 10; R2 is 10 for complete transmission and -15 for packet loss. The closed-loop decision model uses a forgetting factor λ=0.7 and a discount factor γ=0.9 to update the Q value.

[0009] Optionally, the deep low-power control module enters deep sleep mode by calling the esp_deep_sleep_start() interface of ESP32-C3.

[0010] Optionally, the intelligent data processing module employs the lightweight LZ77 compression algorithm, dynamically adjusting the window size and precision based on the compression ratio determined, as follows: At 0% compression, the original data is transmitted, with precision retained to 2 decimal places; With 30% compression, the window size is 8 bytes, and the precision retains 1 decimal place; At 60% compression, the window size is 16 bytes, the precision retains 0 decimal places, and the error rate is less than 3%.

[0011] A low-power transmission method for the Internet of Things (IoT) based on Markov decision-making, the method comprising the following steps: Collect three-dimensional status data of the terminal; the three-dimensional status includes: power status, communication quality, and data priority; Based on the acquired 3D state data, the MDP decision module outputs the optimal action, performs data acquisition, compression, transmission, and sleep, and updates the Q table based on the transmission results. Specifically, this includes: In the MDP decision-making process of the MDP decision module, the forgetting factor is used to prioritize learning new scenario experience, and the discount factor is used to balance the weight of current and future rewards. The LZ77 algorithm is used to dynamically adjust the window size according to the compression ratio, while retaining the preset precision after compression; The terminal enters deep sleep mode immediately after completing data transmission and is only woken up by a timer; The gateway receives data sent by the terminal and forwards it to the cloud. Users can monitor the status and dynamically adjust policy parameters through the cloud GUI. The gateway then sends the new policy to the terminal for execution.

[0012] An electronic device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to enable the electronic device to perform a low-power transmission method for the Internet of Things based on Markov decision-making.

[0013] A computer-readable storage medium storing instructions that, when executed, perform a low-power transmission method for the Internet of Things based on Markov decision-making.

[0014] The beneficial effects of the technical solution provided in this application are: 1. An innovative three-dimensional state space of power, communication quality, and data priority was constructed. Combined with a lightweight MDP model that incorporates a forgetting factor, the terminal can perceive the environment in real time and dynamically adjust the reporting cycle and compression ratio, breaking through the limitations of traditional fixed parameter strategies.

[0015] 2. The LZ77 lightweight compression algorithm is deeply integrated with the ESP32-C3 hardware deep sleep mode, and the ultra-low power state is adaptively triggered according to the decision results, realizing end-to-end power optimization from data generation to transmission.

[0016] 3. A dynamically configurable reward function with weights was designed. Through a three-layer architecture of terminal-gateway-cloud, a dual-mode system of terminal autonomous decision-making and cloud remote control was achieved, balancing system adaptability and scenario controllability. The technical solution of this invention significantly improves the terminal's battery life, while ensuring data transmission reliability through dynamic strategies. It adapts low-computing-power algorithms to low-cost hardware, possessing extremely high engineering application value. Attached Figure Description

[0017] The present application will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a system architecture diagram in an embodiment of this application; Figure 2 This is a flowchart illustrating the process in the embodiments of this application; Figure 3 This is a schematic diagram of the electronic device structure in the embodiments of this application. Detailed Implementation

[0018] To provide a clearer understanding of the technical features, objectives, and effects of this application, the specific embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0019] The embodiments of this application provide a low-power transmission method for the Internet of Things based on Markov decision-making.

[0020] Please refer to Figure 1 , Figure 1 This is a system architecture diagram of a Markov decision-based low-power transmission system for the Internet of Things, as described in an embodiment of this application, including: The terminal layer, gateway layer, and cloud layer are connected in sequence, with the terminal layer and cloud layer being connected. The terminal layer includes: a multi-dimensional state perception module, an MDP dynamic strategy decision-making module, a deep low-power control module, and an intelligent data processing module. A multi-dimensional status awareness module is used to collect the terminal's power status, communication quality, and data priority in real time. The MDP dynamic strategy decision module is used to select the optimal transmission action based on the output of the multi-dimensional state perception module and the lightweight MDP algorithm. A deep low-power control module is used to control the terminal to enter a deep sleep mode based on the output of the decision module. The intelligent data processing module is used to compress the collected data based on the decision results; The gateway layer includes: a dual communication protocol adaptation module, a data relay and caching module, and a policy distribution module; A dual communication protocol adapter module is used to enable LoRa communication between the terminal layer and the gateway layer and WiFi communication between the gateway layer and the cloud layer. The data relay and caching module is used to receive terminal data and cache it to local storage; The policy delivery module is used to receive policy parameters delivered by the cloud layer and broadcast them to the terminal. The cloud layer includes: a strategy parameter configuration module, a running status monitoring module, and a data storage and analysis module; The strategy parameter configuration module is used to configure and distribute MDP strategy parameters; The operation status monitoring module is used to monitor the terminal's operation status and issue abnormal alarms; The data storage and analysis module is used to store historical data and perform analysis.

[0021] This application provides an embodiment as follows: the gateway layer acts as a "communication bridge" to achieve seamless conversion between LoRa and WiFi protocols: Dual communication protocol adaptation module: SX1278 module (202) communicates with the terminal via LoRa (frequency 433MHz, spreading factor 12), WiFi module (201) communicates with Alibaba Cloud MQTT (port 1883, heartbeat cycle 60s); Data relay and caching module: after receiving terminal data, it parses it into JSON format and adds the terminal ID and timestamp, and caches it in local Flash (capacity 4MB). When WiFi is disconnected, it caches ≤1000 data entries and uploads them in batches after recovery; Policy distribution module: it receives weight parameters (such as reward function weight adjustment) distributed by the cloud GUI and broadcasts them to all terminals via LoRa to ensure that the policy is updated synchronously.

[0022] As one example, the cloud layer provides visual configuration and monitoring capabilities, with the core being a host computer GUI policy configuration tool (developed based on PyQt5): Policy parameter configuration module: provides visual adjustment of parameters such as "reporting cycle range," "compression ratio level," and "reward function weight," supporting one-click deployment to the gateway; Operational status monitoring module: displays real-time battery power, signal strength, current policy, and data transmission success rate for each terminal, supporting abnormal alarms (such as terminal offline for >1 hour); Data storage and analysis module: connects to Alibaba Cloud OSS storage, retains one year of historical data, and supports statistical analysis of indicators such as power consumption trends and data integrity.

[0023] As one embodiment, the key points of the present invention are as follows: 1. Three-dimensional state-driven MDP decision architecture: Innovatively constructs a three-dimensional state space of "power-signal-data priority", breaking through the limitations of traditional single-dimensional decision-making. Combined with the forgetting factor, it enables rapid strategy iteration in dynamic environments, with low decision latency and adaptability to low computing power terminals. 2. Hardware-software integrated low-power design: The terminal layer integrates the "LZ77 lightweight compression algorithm" and the "ESP32-C3 deep sleep mechanism", which significantly improves the battery life compared with traditional solutions and solves the core pain point of "long battery life" for IoT terminals; 3. Three-layer architecture collaborative communication mechanism: The gateway layer implements LoRa and WiFi protocol bridging and local caching, the cloud layer provides visual policy configuration, and the terminal layer autonomously executes decisions. The entire link supports a dual mode of "terminal autonomous decision-making + cloud remote control", taking into account both adaptability and controllability. 4. Reward function with dynamic weight configuration: The reward weights for power consumption and integrity can be adjusted in real time through the cloud GUI without modifying the terminal code, adapting to different scenario requirements.

[0024] The core advantages of this invention are: breaking through the limitations of traditional fixed parameters, achieving dynamic environment adaptation through MDP three-dimensional decision-making; realizing coordinated low power consumption of hardware hibernation and software compression, significantly improving battery life; adopting a three-layer architecture design, taking into account terminal autonomous decision-making and cloud control, and being universally adaptable to multiple scenarios; and optimizing lightweight algorithms to adapt to ESP32 series low computing power terminals, possessing high engineering value.

[0025] In the multi-dimensional state perception module: The battery status is divided into five levels based on the battery voltage; The communication quality is divided into four levels based on RSSI signal strength: Excellent, Good, Average, and Poor. The data priority is divided into four levels based on the fluctuation range of temperature and humidity: over-threshold warning, high fluctuation, stable and low change.

[0026] As one embodiment, the terminal's three-dimensional status is collected in real time by three types of sensors to provide data input for MDP decision-making: Battery status: Battery voltage is collected through VBAT pin + ADC calibration and divided into 5 levels (100% ≥ 3.7V, 80% = 3.6V, 60% = 3.5V, 40% = 3.4V, ≤20% < 3.3V); Communication quality: RSSI signal strength is read through SX1278 module and divided into 4 levels (≥ -50dBm Excellent, -70~-50dBm Good, -90~-70dBm Medium, ≤ -90dBm Poor); Data priority: Based on the temperature and humidity fluctuation range collected by DHT22, it is divided into 4 levels (Over-threshold warning > ±5℃ / 10%RH, High fluctuation ±3~5℃ / 5~10%RH, Stable ±1~3℃ / 2~5%RH, Low change < ±1℃ / 2%RH).

[0027] The MDP dynamic strategy decision module specifically includes: By optimizing the lightweight MDP algorithm and the forgetting factor, a closed-loop decision-making model for state-action rewards is constructed, namely the MDP dynamic policy decision module. The state space of the closed-loop decision model consists of a combination of three-dimensional states: power status, communication quality, and data priority, resulting in a total of 80 discrete states. The action space of the closed-loop decision model consists of a two-dimensional combination of reporting period and compression ratio. The reporting period includes four levels: 10 seconds, 30 seconds, 120 seconds, and 900 seconds. The compression ratio includes three levels: 0%, 30%, and 60%, for a total of 12 actions. The reward function R of the closed-loop decision model is composed of a weighted sum of power consumption reward R1 and data integrity reward R2, with the specific formula as: R = 0.6R1 + 0.4R2, where R1 = (baseline power consumption - actual power consumption) / baseline power consumption * 10; R2 is 10 for complete transmission and -15 for packet loss. The closed-loop decision model uses a forgetting factor λ=0.7 and a discount factor γ=0.9 to update the Q value.

[0028] The deep low-power control module enters deep sleep mode by calling the esp_deep_sleep_start() interface of ESP32-C3.

[0029] As one example, the deep low-power control module triggers hardware-level sleep based on the decision result. The core is the deep_sleep mode adaptation of ESP32-C3: after the policy is executed, the esp_deep_sleep_start() interface is called to retain only the timer wake-up source.

[0030] The intelligent data processing module employs the lightweight LZ77 compression algorithm, dynamically adjusting the window size and precision based on the compression ratio determined, as follows: At 0% compression, the original data is transmitted, with precision retained to 2 decimal places; With 30% compression, the window size is 8 bytes, and the precision retains 1 decimal place; At 60% compression, the window size is 16 bytes, the precision retains 0 decimal places, and the error rate is less than 3%.

[0031] Please refer to Figure 2 , Figure 2 This is a flowchart of a Markov decision-based low-power transmission method for the Internet of Things (IoT) according to an embodiment of this application, including the following steps: Collect three-dimensional status data of the terminal; the three-dimensional status includes: power status, communication quality, and data priority; Based on the acquired 3D state data, the MDP decision module outputs the optimal action, performs data acquisition, compression, transmission, and sleep, and updates the Q table based on the transmission results. Specifically, this includes: In the MDP decision-making process of the MDP decision module, the forgetting factor is used to prioritize learning new scenario experience, and the discount factor is used to balance the weight of current and future rewards. The LZ77 algorithm is used to dynamically adjust the window size according to the compression ratio, while retaining the preset precision after compression; The terminal enters deep sleep mode immediately after completing data transmission and is only woken up by a timer; The gateway receives data sent by the terminal and forwards it to the cloud. Users can monitor the status and dynamically adjust policy parameters through the cloud GUI. The gateway then sends the new policy to the terminal for execution.

[0032] This application provides an embodiment as follows: the cloud-based policy configuration allows users to dynamically adjust the reward function weight, reporting cycle range, and compression ratio level through a GUI interface, enabling remote updates and adaptation of the policy without modifying the terminal code.

[0033] This application also discloses an electronic device. (See reference...) Figure 3 , Figure 3 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. The electronic device 500 may include: at least one processor 501, at least one network interface 504, a user interface 503, a memory 505, and at least one communication bus 502.

[0034] The communication bus 502 is used to enable communication between these components.

[0035] The user interface 503 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. The network interface 504 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0036] This application also discloses a computer-readable storage medium storing multiple instructions adapted for loading by a processor to execute the aforementioned low-power transmission method for the Internet of Things based on Markov decision-making.

[0037] The above are merely exemplary embodiments of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure.

[0038] This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described in this disclosure. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

Claims

1. A Markov decision based low power transmission system for Internet of Things, characterized in that, The system comprises: a terminal layer, a gateway layer, and a cloud layer connected sequentially, wherein the terminal layer and the cloud layer are connected. The terminal layer includes: a multi-dimensional state awareness module, an MDP dynamic policy decision-making module, a deep low-power control module, and an intelligent data processing module. The multi-dimensional state awareness module is used to collect the terminal's power status, communication quality, and data priority in real time. The MDP dynamic policy decision-making module is used to select the optimal transmission action based on a lightweight MDP algorithm according to the output of the multi-dimensional state awareness module. The deep low-power control module is used to control the terminal to enter a deep sleep mode according to the output of the decision-making module. The intelligent data processing module is used to compress the collected data according to the decision results. The gateway layer includes: The system comprises a dual communication protocol adaptation module, a data relay and caching module, and a policy distribution module. The dual communication protocol adaptation module enables LoRa communication between the terminal layer and the gateway layer, and WiFi communication between the gateway layer and the cloud layer. The data relay and caching module receives terminal data and caches it locally. The policy distribution module receives policy parameters from the cloud layer and broadcasts them to the terminal. The cloud layer includes a policy parameter configuration module, a runtime monitoring module, and a data storage and analysis module. The policy parameter configuration module configures and distributes MDP policy parameters. The runtime monitoring module monitors the terminal's runtime status and issues abnormal alarms. The data storage and analysis module stores historical data and performs analysis.

2. The Markov decision based low power transmission system for IoT of claim 1, wherein, In the multi-dimensional state perception module: the power status is divided into five levels based on battery voltage; the communication quality is divided into four levels based on RSSI signal strength, specifically: excellent, good, medium, and poor; the data priority is divided into four levels based on the temperature and humidity fluctuation amplitude, specifically: over-threshold warning, high fluctuation, stable, and low change.

3. The Markov decision based low power transmission system for IoT of claim 1, wherein, The MDP dynamic policy decision module specifically includes: constructing a closed-loop decision model of state-action reward through a lightweight MDP algorithm and forgetting factor optimization, namely the MDP dynamic policy decision module; the state space of the closed-loop decision model is composed of a three-dimensional combination of power status, communication quality and data priority, with a total of 80 discrete states; the action space of the closed-loop decision model is composed of a two-dimensional combination of reporting period and compression ratio, wherein the reporting period includes four levels: 10 seconds, 30 seconds, 120 seconds and 900 seconds, and the compression ratio includes three levels: 0%, 30% and 60%, with a total of 12 actions; The reward function R of the closed-loop decision model is composed of a weighted sum of power consumption reward R1 and data integrity reward R2, with the specific formula: R = 0.6 * R1 + 0.4 * R2, where R1 = (baseline power consumption - actual power consumption) / baseline power consumption * 10; R2 is 10 when the transmission is complete and -15 when there is packet loss. The closed-loop decision model uses a forgetting factor λ=0.7 and a discount factor γ=0.9 to update the Q value.

4. The Markov decision based low power transmission system for IoT of claim 1, wherein, The deep low-power control module enters deep sleep mode by calling the esp_deep_sleep_start() interface of ESP32-C3.

5. The Markov decision based low power transmission system for IoT of claim 3, wherein, The intelligent data processing module employs the lightweight LZ77 compression algorithm, dynamically adjusting the window size and precision based on the compression ratio determined, as follows: At 0% compression, the original data is transmitted, with precision retained to 2 decimal places; With 30% compression, the window size is 8 bytes, and the precision retains 1 decimal place; At 60% compression, the window size is 16 bytes, the precision retains 0 decimal places, and the error rate is less than 3%.

6. A Markov decision-based low-power transmission method for Internet of Things, based on the Markov decision-based low-power transmission system for Internet of Things according to any one of claims 1-5, characterized in that, The method includes the following steps: collecting three-dimensional status data of the terminal; the three-dimensional status includes: battery status, communication quality, and data priority; based on the collected three-dimensional status data, the optimal action is output through the MDP decision module, performing data collection, compression, transmission, and sleep, and updating the Q table according to the transmission results, specifically including: in the MDP decision process of the MDP decision module, the forgetting factor is used to prioritize learning new scenario experience, and the discount factor is used to balance the weight of current and future rewards; the LZ77 algorithm is used to dynamically adjust the window size according to the compression ratio, and the preset precision is retained after compression; the terminal immediately enters deep sleep mode after completing data transmission and is only woken up by a timer; the gateway receives the data sent by the terminal and forwards it to the cloud, the user monitors the status and dynamically adjusts the policy parameters through the cloud GUI, and the gateway sends the new policy to the terminal for execution.

7. An electronic device, comprising: The device includes a processor, a memory, a user interface, and a network interface. The memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory to enable the electronic device to perform the low-power transmission method for the Internet of Things based on Markov decision-making as described in claim 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed by a computer, perform the low-power transmission method for the Internet of Things based on Markov decision-making as described in claim 6.

Images