An iot data adaptive collection and packaging method and system thereof

Abstract

This invention discloses an adaptive data acquisition, signing, and encapsulation method and system for the Internet of Things (IoT). The method includes: dynamically determining a batch signing threshold based on environmental parameters; acquiring data in real time, calculating and caching hash values, and encapsulating the data and its corresponding hash values ​​into data packets for transmission to the cloud; when the number of transmitted data packets reaches the batch signing threshold, triggering a signing operation, generating an overall digital signature for all cached hash values ​​using a built-in private key, and encapsulating the set of all hash values ​​and the overall digital signature into a signature packet for transmission to the cloud for cloud verification of data integrity and batch confirmation of data packet authenticity. This invention changes the traditional strongly bound encapsulation mode of "data + signature," separating the data packets and signature packets into separate encapsulations, changing the traditional "signature for every packet" to "acquisition and transmission as needed, batch signing," greatly reducing the frequency of asymmetric encryption by hardware security chips, eliminating transmission congestion caused by encryption, and significantly increasing system throughput.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) data acquisition technology, specifically to an IoT data adaptive acquisition and encapsulation method and system. Background Technology

[0002] In Internet of Things (IoT) applications, such as smart grids, industrial sensor networks, and smart cities, a massive number of edge devices need to continuously report data to the cloud or edge computing nodes. Because these devices are typically deployed in open, unattended physical environments, and communication links such as NB-IoT, LoRa, and cellular networks are mostly public wireless networks, the data is highly vulnerable to eavesdropping, tampering, forgery, and replay attacks during transmission.

[0003] To ensure data security, the industry typically employs the following data encapsulation and secure transmission technologies:

[0004] 1. Transport layer encrypted tunnel (such as TLS / DTLS): Establish a certificate-based secure tunnel between edge devices and cloud servers, and encrypt and protect the integrity of all transmitted data at the underlying layer.

[0005] 2. Application Layer Packet Signature Encapsulation (Data+Hash+Signature): At the application layer, for each packet of valid data collected by the sensor, the edge device first calculates the hash value, then calls an asymmetric encryption algorithm such as RSA or SM2 (China's national standard) to digitally sign the hash value using the device's private key, and finally packages the "original data + hash value + digital signature" into a complete data packet and sends it to the receiving end.

[0006] 3. Introduction of hardware security chip (SE / TPM): In order to prevent the device's private key from being stolen by hackers through firmware reverse engineering or memory dumping, high-security IoT terminals will introduce independent hardware security chips to isolate key generation, storage and cryptographic operations (Hash calculation and signature calculation) within the security chip.

[0007] While the aforementioned technologies can provide high data security, they have the following drawbacks on IoT devices with limited computing resources, battery power, and narrow network bandwidth:

[0008] 1. Extremely high computational overhead, leading to a surge in power consumption and processing bottlenecks: Asymmetric encryption, such as digital signatures, is an extremely computationally intensive operation. In the "packet-by-packet signature" mode, IoT devices must perform a signature operation every time they send a packet of data. However, IoT terminals are typically microcontrollers (MCUs), whose main frequency is usually only tens of MHz. The high frequency of signatures will quickly deplete the device's battery power. At the same time, frequent calls to the hardware security chip to perform signatures can cause blockages in the data processing pipeline, making it impossible to meet the data acquisition needs of high-frequency sensors.

[0009] 2. Excessive signature payload severely wastes network bandwidth and causes latency: IoT sensor data itself is often very small, such as a few bytes of temperature value, but an asymmetric digital signature is usually tens or even hundreds of bytes long, such as an ECC signature of about 64 bytes and an RSA signature of up to 256 bytes. If every packet is accompanied by a signature, the size of the security payload will far exceed the business data itself. In narrowband networks (such as LoRa and NB-IoT), this will cause the transmission time to increase exponentially, and may even force fragmentation due to exceeding the maximum transmission unit (MTU) of a single packet, thus greatly increasing network latency.

[0010] 3. Lack of environmental awareness: Existing security encapsulation strategies are static, such as using fixed per-packet signing or a fixed number of packets. The system cannot perceive the current network congestion, remaining device battery power, or the urgency of the current data. For example, in environments with extremely poor network conditions and high packet loss rates, adhering to the original heavy-load signing strategy will cause numerous transmission timeouts; and for high-priority fire alarm data, if it is still mixed with regular data and packaged according to fixed logic, it will lead to fatal alarm delays.

[0011] 4. Inefficient retransmission mechanism in weak network environment: In the traditional "data + signature" strong binding encapsulation scheme, once a small network jitter causes a data packet to be lost or a byte to be corrupted, the terminal device must recalculate and retransmit the entire data packet containing the huge signature after the receiving end fails to verify it. This further aggravates network congestion and will cause a vicious cycle of "network congestion - frequent packet loss - repeated retransmission" in a weak network environment. Summary of the Invention

[0012] The purpose of this invention is to provide an adaptive data collection and encapsulation method for the Internet of Things (IoT) to solve the problems mentioned in the background art.

[0013] To achieve the above objectives, the present invention provides the following technical solution: an adaptive tagging and encapsulation method for Internet of Things (IoT) data, the method comprising:

[0014] The batch signature threshold is dynamically determined based on environmental parameters.

[0015] Collect data in real time, calculate hash values ​​and cache them, and encapsulate the data and its corresponding hash values ​​into data packets and send them to the cloud.

[0016] When the number of data packets sent reaches the batch signature threshold, a signature operation is triggered. The built-in private key is used to generate an overall digital signature for all cached hash values. The set of all hash values ​​and the overall digital signature are then packaged into a signature packet and sent to the cloud for cloud verification of data integrity. The overall data signature is verified using the public key to confirm the authenticity of the data packets in batches.

[0017] Furthermore, the cloud-based verification of data integrity and the batch confirmation of data packet authenticity through overall data signature include:

[0018] After receiving the data packet, the cloud recalculates the hash value and compares it with the hash value attached to the data packet to verify data integrity.

[0019] After receiving the signature packet in the cloud, the cloud uses the public key to verify the overall digital signature. If the verification is successful, the authenticity of the data packets is confirmed in batches.

[0020] Furthermore, the cloud also initiates a retransmission request based on the missing information;

[0021] If a missing data packet is detected, based on the hash value stored in the signature packet, only the missing original data is requested to be retransmitted;

[0022] If a missing signature packet is detected, only request a retransmission of the signature packet.

[0023] Furthermore, the environmental parameters include one or more of network quality, device power consumption, and data priority.

[0024] Furthermore, when the network quality is better than the first preset threshold, the battery level is higher than the second preset threshold, and the data is of normal priority, a first batch signature threshold is set.

[0025] When the network packet loss rate is higher than the third preset threshold or the signal strength is lower than the fourth threshold, the batch signature threshold is reduced to the second batch signature threshold, which is less than the first batch threshold and greater than 1.

[0026] When alarm-type or high-priority burst data is detected, the batch signature threshold is forcibly set to 1.

[0027] Furthermore, the step of generating an overall digital signature using the built-in private key includes: concatenating the hash values ​​corresponding to the collected data according to the order in which the data packets were generated to form a sequence set, and signing the sequence set to generate an overall digital signature.

[0028] Furthermore, the step of generating an overall digital signature using the built-in private key includes: constructing a hash tree, using the hash values ​​corresponding to the collected data as leaf nodes, pairing them up to calculate the upper-level hash values, recursively calculating layer by layer until the root hash value is calculated, and then signing the root hash value to generate an overall digital signature.

[0029] Furthermore, after the signature packet is sent to the cloud, the cached hash value is cleared to prepare for the next signing cycle.

[0030] An adaptive data acquisition and packaging system for the Internet of Things (IoT), the system comprising an IoT terminal and a cloud server;

[0031] The IoT terminal includes:

[0032] The control module is used to monitor environmental parameters in real time and dynamically determine the batch signature threshold based on the monitoring results.

[0033] The data acquisition module is used to collect data in real time.

[0034] The hardware security chip has a built-in, non-exportable device private key, which is used to calculate the hash value of the collected data and cache it in the internal secure storage area. When the batch signature threshold is reached, the device private key is used to generate an overall digital signature for all the cached hash values.

[0035] The encapsulation and transmission module is used to encapsulate each piece of data and its corresponding hash value into a data packet and send it to the cloud in real time. When the batch signature threshold is reached, the set of all hash values ​​and the overall digital signature are encapsulated into a signature packet and sent to the cloud.

[0036] The cloud server is used to receive data packets and signature packets, verify data integrity, verify the overall data signature using a public key, and confirm the authenticity of data packets in batches.

[0037] Furthermore, the hardware security chip is a Security Element (SE) or a Trusted Platform Module (TPM), which integrates a cryptographic acceleration engine for high-speed hash calculations.

[0038] Compared with the prior art, the beneficial effects of the present invention are:

[0039] 1. This invention changes the traditional strong-binding encapsulation mode of "data + signature" by separating the data packet and the signature packet into separate encapsulations. It changes the traditional "sign every packet must be signed" to "send as you collect, sign in batches", which greatly reduces the frequency of asymmetric encryption of hardware security chips, eliminates transmission blockage caused by encryption, and multiplies the system throughput. It breaks through the static limitation of fixed batch signing and can dynamically adjust the batch signing threshold according to environmental parameters. Without sacrificing data integrity and non-repudiation, it minimizes the signature calculation overhead of IoT devices and saves network bandwidth, achieving an adaptive optimal solution of "saving power and bandwidth for regular data and ensuring real-time and security for emergency data".

[0040] 2. The entire process of hash value calculation, caching, and private key signing is completed within the physical security boundary of the hardware security chip. The device private key cannot be exported from the date of generation. All cryptographic operations are performed in a closed loop inside the chip, which resists firmware reverse analysis, runtime memory theft, and physical side-channel attacks targeting the IoT main control MCU, ensuring the authenticity of the device identity and the absolute security of the signing key.

[0041] 3. By decoupling the hash values ​​in the data packets and signature packets, an asymmetric retransmission mechanism is adopted. If a data packet is lost, only the original data needs to be retransmitted, without retransmitting other data and its signature. If a missing signature packet is lost, only a small signature packet needs to be retransmitted, without retransmitting massive amounts of original data. Compared with the full retransmission of "data + signature" in traditional solutions, this significantly reduces invalid data transmission in weak network environments such as narrowband IoT, and significantly reduces bandwidth consumption and terminal power consumption caused by retransmission. Attached Figure Description

[0042] Figure 1 This is a block diagram of the IoT data adaptive acquisition and packaging system of the present invention;

[0043] Figure 2 This is a flowchart of the IoT data adaptive collection and encapsulation method of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] like Figure 1 As shown, this embodiment provides an IoT data adaptive collection and packaging system, the system including an IoT terminal 100 and a cloud server 200;

[0046] The IoT terminal 100 includes a control module 1, a data acquisition module 2, a hardware security chip 3, and an encapsulation and transmission module 4.

[0047] The control module 1 runs on a microcontroller (MCU) and is used to monitor environmental parameters in real time. Based on the monitoring results, it dynamically determines the batch signature threshold n. In this embodiment, it monitors the network quality (e.g., packet loss rate, RSSI signal strength), battery level, and data priority of the current device in real time, dynamically calculates and outputs the current batch signature threshold n. When the network quality is better than a first preset threshold, the battery level is higher than a second preset threshold, and the data has a normal priority, the first batch signature threshold is set. When the network packet loss rate is higher than a third preset threshold or the signal strength is lower than a fourth threshold, the batch signature threshold is reduced to a second batch signature threshold, which is less than the first batch threshold and greater than 1. When alarm-type or high-priority burst data is detected, the batch signature threshold is forcibly set to 1. For example, if... If the current network quality is good, the device power is normal, and the data is routine, a larger n is set, such as n=50; if the network packet loss rate is high, n is reduced, such as n=10; if high-priority burst data such as alarms is collected, n is forcibly set to 1. This strategy can sense network quality, device power, and business priority to dynamically adjust the batch signature threshold n, breaking through the static limitations of fixed batch signatures. It can dynamically adjust the batch signature threshold according to environmental parameters, minimizing the signature calculation overhead of IoT devices and saving network bandwidth without sacrificing data integrity and non-repudiation. It achieves an adaptive optimal solution of "saving power and bandwidth for routine data and ensuring real-time and security for emergency data". For example, fire alarm data can be instantly converted into data collected and signed, balancing security and performance and ensuring zero-delay on-chain uploading of critical events.

[0048] The data acquisition module 2 is used to acquire data in real time, such as acquiring physical data from sensors, including temperature, humidity, vibration, current harmonics, etc.

[0049] The hardware security chip 3 is a Security Unit (SE) or Trusted Platform Module (TPM), a physically independent chip that integrates a cryptographic acceleration engine for high-speed hash calculations. It contains a built-in, non-exportable device private key responsible for high-speed hash calculations and asymmetric digital signatures. It calculates hash values ​​for the collected data and caches them in its internal secure storage area. When the batch signature threshold n is reached, it generates an overall digital signature from all cached hash values ​​using the device private key. For example, it generates a digital signature for the i-th data collected by the data acquisition module 2. Perform a hash operation on (1 ≤ i ≤ n) to generate a Hash( The chip caches this hash value internally. When the number of sent data packets reaches the batch signature threshold n, a signature operation is triggered, and the internally cached n hash values ​​are hashed ( ) to Hash( The hashes are concatenated or a hash tree is constructed, and a single digital signature is performed using the built-in private key. This process completes the entire process of hash value calculation, caching, and private key signing within the physical security boundary of the hardware security chip. The device private key cannot be exported from the date of its generation, and all cryptographic operations are performed in a closed loop within the chip. This resists firmware reverse analysis, runtime memory theft, and physical side-channel attacks targeting the IoT main control MCU, ensuring the authenticity of the device identity and the absolute security of the signing key.

[0050] The encapsulation and transmission module 4 is used to encapsulate each piece of data and its corresponding hash value into a data packet and send it to the cloud server 200 in real time. When the batch signature threshold n is reached, it encapsulates the set of all hash values ​​and the overall digital signature into a signature packet and sends it to the cloud server 200. For example, the encapsulation and transmission module 4 encapsulates the data... With the corresponding Hash( The data is encapsulated into a data packet and immediately sent to the cloud server 200. When the number of data packets sent reaches the batch signature threshold n, the set of these n hash values ​​and the overall digital signature are extracted. The data is packaged into an independent signature packet and sent to the cloud server 200. This process changes the traditional strong binding encapsulation mode of "data + signature". The data packet and the signature packet are separated and decoupled. The data packet only contains "data + the hash corresponding to the data" for rapid data reporting and ensuring the integrity of a single packet. The signature packet only contains "a set of hashes of multiple data packets + a unified digital signature" to provide batch non-repudiation proof for n data packets. The traditional "signing every packet" is changed to "sending as soon as it is collected, signing in batches", which compresses n signatures into 1. This greatly reduces the frequency of asymmetric encryption of the hardware security chip, which can be reduced to 1 / n of the original. It eliminates the transmission blockage caused by encryption and increases the system throughput many times. Moreover, the cloud server 200 only needs to verify once to confirm the integrity and source credibility of the entire batch of data during signature verification.

[0051] The cloud server 200 is used to receive data packets and signature packets, verify data integrity, verify the overall data signature using a public key, and confirm the authenticity of data packets in batches. In this embodiment, the cloud server 200 receives data packets, verifies integrity by recalculating the hash value of the data and comparing it with the hash value attached to the data packet; it receives signature packets and verifies the signature using the device's public key. If successful, the authenticity and non-repudiation of these n data packets are confirmed in batches. If the cloud discovers that the kth data packet is missing, the signature packet contains a hash ( The cloud can request a retransmission individually. This does not affect the on-chaining and verification of the other n-1 data. If the cloud detects a lost signature packet, it only requires the terminal to retransmit the signature packet. The terminal does not need to resend the massive original data packets 1 to n. In this invention, by using the decoupling design of the data packet and the hash value in the signature packet, an asymmetric retransmission mechanism is adopted. If a data packet is lost, only the original data needs to be retransmitted, and other data and their signatures do not need to be retransmitted. If a missing signature packet is lost, only a smaller signature packet needs to be retransmitted, and massive amounts of original data do not need to be retransmitted. Compared with the full retransmission of "data + signature" in the traditional solution, in weak network environments such as narrowband IoT, invalid data transmission is greatly reduced, and the bandwidth consumption and terminal power consumption caused by retransmission are significantly reduced.

[0052] This embodiment also provides an adaptive tagging and encapsulation method for IoT data, the method comprising the following steps:

[0053] S1: Dynamically determine the batch signature threshold based on environmental parameters;

[0054] S2: Collect data in real time, calculate hash values ​​and cache them, and encapsulate the data and its corresponding hash values ​​into data packets and send them to the cloud;

[0055] S3: When the number of data packets sent reaches the batch signature threshold, a signature operation is triggered. The built-in private key is used to generate an overall digital signature for all cached hash values. The set of all hash values ​​and the overall digital signature are then encapsulated into a signature packet and sent to the cloud for cloud verification of data integrity. The overall data signature is verified using the public key to confirm the authenticity of the data packets in batches.

[0056] Specifically, step 1, adaptive batch signature threshold setting:

[0057] Control module 1 monitors the network quality (e.g., packet loss rate, RSSI signal strength), battery level, and data priority of the current device in real time. It dynamically calculates and outputs the current batch signature threshold n. When the network quality is better than the first preset threshold, the battery level is higher than the second preset threshold, and the data is of normal priority, the first batch signature threshold is set. When the network packet loss rate is higher than the third preset threshold or the signal strength is lower than the fourth threshold, the batch signature threshold is reduced to the second batch signature threshold, which is less than the first batch threshold and greater than 1. When alarm-type or high-priority burst data is detected, the batch signature threshold is forcibly set to 1. For example, if the current device has good network quality, normal battery level, and normal data... Set a larger n, such as n=50; if the network packet loss rate is high, decrease n, such as n=10; if high-priority burst data such as alarms is collected, force n=1; this strategy can sense network quality, device power consumption and business priority to dynamically adjust the batch signature threshold n, breaking through the static limitation of fixed batch signature. It can dynamically adjust the batch signature threshold according to environmental parameters, and minimize the signature calculation overhead of IoT devices and save network bandwidth without sacrificing data integrity and non-repudiation. It achieves the adaptive optimal solution of "saving power and bandwidth for regular data and ensuring real-time and security for emergency data". For example, fire alarm data can be instantly converted into instant signature, balancing security and performance, and ensuring zero-delay on-chain uploading of critical events.

[0058] Step 2, Data Packet Generation and Sending:

[0059] Data acquisition module 2 collects the i-th data. (1 ≤ i ≤ n), 3 pairs of hardware security chips Perform a hash operation to generate a Hash( The hash value is cached inside the chip, and the encapsulated transmission module 4 encapsulates the data. With the corresponding Hash( The data is encapsulated into a data packet and immediately sent to the cloud server 200. At this time, the cloud server 200 can only verify whether the data is corrupted through the hash value, but has not yet verified the identity.

[0060] Step 3: Generate and send the signature packet:

[0061] When the number of data packets sent reaches the batch signature threshold n, a signature operation is triggered. The hardware security chip 3 hashes the n internally cached hash values. ) to Hash( The hashes are concatenated or a hash tree is constructed, and a single digital signature is performed using the built-in private key. For example, the hash values ​​corresponding to the collected data can be concatenated according to the order in which the data packets were generated to form a sequence set. The sequence set can then be signed to generate an overall digital signature. For the sake of simplicity, assume that n=4, that is, the hardware security chip 3 internally caches the hash values ​​of 4 data packets: H1 = SHA256(D1) = a1b2c3... (32 bytes), H2 = SHA256(D2) = d4e5f6... (32 bytes), H3 = SHA256(D3) = 7g8h9i... (32 bytes), H4 = SHA256(D4) = j0k1l2... (32 bytes);

[0062] They are concatenated in order into a 128-byte (32*4) data block: H_combined = H1 || H2 || H3 || H4 (|| represents concatenation), i.e.: a1b2c3...d4e5f6...7g8h9i...j0k1l2... Using the internally stored, non-exportable device private key, such as the SM2 private key, a digital signature algorithm is executed on the 128-byte data block H_combined to generate a signature result Sign_{1~4}, such as a 64-byte SM2 signature. This Sign_{1~4} represents its authoritative authentication of the hash set H1, H2, H3, and H4.

[0063] When n is large, direct concatenation will result in a very long data block to be signed. In this case, a hash tree can be constructed, with the hash value corresponding to the collected data as the leaf node. The upper-level hash value is calculated by pairing them up and down, and so on, until the final root hash value is calculated. The root hash value is then signed to generate the overall digital signature. Assuming n=4, there are four data packets with hash values ​​H1, H2, H3, and H4 as leaf nodes. H12 = Hash(H1 || H2) and H34 = Hash(H3 || H4) are calculated. Here, Hash can be the same hash function (such as SHA256). The root hash value Root = Hash(H12 || H34) is calculated. The hardware security chip 3 uses the device private key to sign only this fixed-length (such as 32 bytes) root hash value Root to generate Sign_Root. Therefore, as long as Sign_Root is verified, the integrity and batch authenticity of all leaf hash values ​​(i.e., H1, H2, H3, H4) under it can be guaranteed. Whether using the "concatenation" or "hash tree construction" method, the hash digests of n data packets are reduced to one (or a group) of data that can represent them as a whole. Then, the hardware security chip 3 performs a single overall digital signature operation to authenticate it, thereby reducing the overhead of n signatures to one.

[0064] When the number of data packets sent reaches the batch signature threshold n, the encapsulation and transmission module 4 extracts the set of these n hash values ​​and the overall digital signature, such as Sign_{1~4} or Sign_Root as mentioned above, encapsulates it into an independent signature packet, and sends it to the cloud server 200. Taking Sign_{1~4} as an example, the signature packet is designed with a simple structure including a header and a payload, as shown in the table below:

[0065]

[0066] The encapsulation and transmission module 4 frames the above fields according to a predetermined format to form a complete binary data packet, which is then sent to the cloud server 200 via a wireless network, such as NB-IoT. This signature packet does not contain any original sensor data, but only their hash digests and the overall digital signature. After transmission is completed, the hardware security chip 3 clears the cache and begins the next round of signature collection. This process places the entire process of hash value calculation, caching, and private key signing within the physical security boundary of the hardware security chip. The device private key cannot be exported from the date of generation, and all cryptographic operations are performed in a closed loop within the chip. This resists firmware reverse analysis, runtime memory theft, and physical side-channel attacks targeting the IoT main control MCU, ensuring the authenticity of the device identity and the absolute security of the signing key. It also changes the traditional strong binding encapsulation mode of "data + signature" by separating the data packet and the signature packet into separate encapsulations. The traditional "sign every packet must be signed" is changed to "send as you collect, sign in batches", which compresses n signatures into 1, greatly reducing the frequency of asymmetric encryption of the hardware security chip to 1 / n of the original. This eliminates the transmission blockage caused by encryption, multiplies the system throughput, and the cloud server 200 only needs to verify once to confirm the integrity and source credibility of the entire batch of data during signature verification.

[0067] Step 4: Cloud server parsing, verification, and retransmission

[0068] The cloud server 200 receives data packets, recalculates the data's hash value, and compares it with the hash value attached to the data packet to verify integrity. If they match, the data is intact; otherwise, the packet is discarded or a retransmission is requested. It also receives a signature packet and verifies the signature using the device's public key. If successful, the authenticity and non-repudiation of these n data packets are confirmed in batches. For example, the overall data signature is decrypted using the public key to obtain the decrypted signature result, and the hash value set corresponding to all data is recalculated to obtain the local calculation result. The decrypted signature result and the local calculation result are compared. If they match, the batch of data packets is genuine; otherwise, they are discarded. If the cloud discovers that the kth data packet is missing, since the signature packet contains a hash ( The cloud can request a retransmission individually. This does not affect the on-chaining and verification of the other n-1 data; if the cloud discovers that the signature packet is lost, the cloud only requires the terminal to retransmit the signature packet, and the terminal does not need to resend the huge original data packets 1 to n.

[0069] The process of receiving and verifying the integrity of a single data packet is as follows: Assume that the encapsulation and transmission module 4 sends data packet 1 (D1, H1'), data packet 2 (D2, H2'), data packet 3 (D3, H3'), data packet 4 (D4, H4'), and a subsequent signature packet (containing [H1, H2, H3, H4] and Sign_{1~4}). For distinction, Hx' represents the hash value attached to the data packet, and Hx represents the hash value calculated by the cloud server 200 or carried in the signature packet. When the cloud server 200 receives data packet 1, it unpacks it to obtain the original data D1 and the attached hash value H1'. It recalculates the hash value of D1 using the same hash algorithm as the terminal (such as SHA256) to obtain H1_cal. It compares H1_cal with the received H1'. If they are equal, it means that D1 has not been tampered with during transmission, and its integrity is verified. The cloud server temporarily stores D1 and H1_cal, waiting for the corresponding signature packet to perform final authentication. However, at this point, it cannot be confirmed that the data definitely comes from a legitimate device and may be forged. If they are not equal, it means that the data was corrupted during transmission, and the cloud can immediately discard the packet or request a retransmission of the data packet.

[0070] Receiving and verifying the signature packet: Assuming the cloud server 200 has correctly received and temporarily stored D1, D2, D3, D4 and their corresponding verified hash values ​​H1_cal, H2_cal, H3_cal, H4_cal, the cloud server 200 receives the signature packet, unpacks it to obtain the hash value set (H1, H2, H3, H4) and the digital signature Sign_{1~4}. The cloud server 200 compares the temporarily stored, integrity-verified hash values ​​(H1_cal, H2_cal, H3_cal, H4_cal) with the hash value set (H1, H2, H3, H4) in the signature packet one by one. If they all match: it means that the data digest to be signed by the terminal is consistent with the complete data digest actually received by the cloud server 200, and proceeds to the next step of signature verification; if they do not match: This indicates that data packets may have malfunctioned during transmission or that signature packets may not have matched, leading to verification failure. During digital signature verification, the IoT terminal uses a public key pre-recorded on the cloud server 200 or issued by a CA to verify the received signatures Sign_{1~4}. The public key is used to decrypt Sign_{1~4}, yielding a decrypted signature result, which is the original data signed by the hardware security chip 3, for example, the concatenated H1||H2||H3||H4. The cloud server 200 also processes the matching hash value set (H1, H2, H3, H4) according to the same rules, such as concatenating them to obtain a local calculation result. The "decrypted signature result" is compared with the "local calculation result." If they match, it proves that this hash set was indeed signed by the entity holding the corresponding private key, i.e., the hardware security chip built into the terminal. This allows for batch confirmation that the original data D1, D2, D3, D4 corresponding to the hash value sets H1, H2, H3, H4 all originate from the legitimate terminal device possessing the private key and have not been tampered with since signing. This simultaneously achieves authenticity (trustworthy source) and non-repudiation (the terminal cannot deny sending this data because only it has the private key to generate the signature). If there is a mismatch: verification fails, and the batch of data is considered suspicious or forged and will be discarded. In this way, the cloud server 200 ensures data integrity (tamper-proof) through "packet-by-packet hash verification" and guarantees the authenticity and non-repudiation of the entire batch of data at once through "batch signature verification." The combination of these two methods significantly reduces the number of signature verification calculations in the cloud from n to 1, while ensuring security.

[0071] Missing Verification Process: Assume the terminal sends data packets 1 (D1, H1'), 2 (D2, H2'), 3 (D3, H3'), and 4 (D4, H4') along with a corresponding signature packet (containing [H1, H2, H3, H4] and the signature Sign_{1~4}) according to the batch signature threshold n=4. Data packet 3 (D3, H3') is lost during network transmission and is not received by cloud server 200. However, cloud server 200 successfully receives data packets 1, 2, and 4, as well as the complete signature packet. The cloud server performs a preliminary integrity check on each received data packet. For successfully received data packet 1 (D1, H1'): the cloud server 200 calculates Hash(D1) to obtain H1_cal and compares it with the attached H1'. If they match, D1 passes the integrity check and (D1, H1_cal) is temporarily stored. Similarly, the same operation is performed on data packets 2 and 4. If both pass the check, (D2, H2_cal) and (D4, H4_cal) are temporarily stored. For the lost data packet 3, the cloud server 200 does not perform any operation and only records its absence. At this point, the cloud server 200 confirms that the received D1, D2, and D4 have not been tampered with during transmission, but it cannot yet confirm whether their source is authentic. Next, the cloud server 200 receives the signature packet and parses it to extract the set of hash values ​​(H1, H2, H3, H4) committed by the terminal and the overall signature Sign_{1~4}. This set of hash values ​​(H1, H2, H3, H4) is the "preimage" calculated and finally signed by the terminal before sending the data, serving as the authoritative basis for verifying the batch. The cloud server 200 performs hash value comparison and matching, comparing the calculated and temporarily stored hash values ​​(H1_cal, H2_cal, H4_cal) with the corresponding hash values ​​(H1, H2, H4) extracted from the signature packet. If H1_cal = H1, H2_cal = H2, and H4_cal = H4, it proves that the hash values ​​D1, D2, and D4 received by the cloud server 200 are completely consistent with the hash values ​​used by the terminal when preparing the signature. This indicates that the content of these data packets has not changed between the generation of hash values ​​by the terminal and the generation of the signature. Next, cloud server 200 performs overall digital signature verification using the public key of the terminal device. The verification process involves decrypting the overall digital signature to obtain the original signed data (i.e., the terminal's processing result of the complete list [H1, H2, H3, H4], such as the concatenated byte string or Merkle root). Cloud server 200 needs to use all the information to "reconstruct" this original signed data. Because data packet 3 corresponding to H3 is missing, cloud server 200 cannot calculate H3 on its own.However, H3 is directly provided in the signature packet. Therefore, the cloud can directly use the complete list [H1, H2, H3, H4] extracted from the signature packet to construct data according to the same rules (such as concatenation) and perform signature verification. If the signature verification passes: it proves that the list [H1, H2, H3, H4] as a whole has obtained the signature of the terminal's private key. Since it has been confirmed that H1_cal = H1, H2_cal = H2, and H4_cal = H4, the signature verification passing means that: H1 is a valid and terminal-certified hash value → therefore D1 is real and non-repudiable; H2 is a valid and terminal-certified hash value → therefore D2 is real and non-repudiable; H4 is a valid and terminal-certified hash value → therefore D4 is real and non-repudiable.

[0072] Therefore, even though packet 3 is lost, the cloud server can still verify the correctness, integrity, authenticity, and non-repudiation of the successfully received packets 1, 2, and 4 with 100% accuracy. This verified data can be delivered to upper-layer applications immediately without waiting for retransmission.

[0073] After completing batch verification, cloud server 200 legally possesses H3 (from the signature packet), but lacks the corresponding original data D3. Based on the information in the signature packet, the cloud server knows that the missing data is serial number 3 and initiates a precise retransmission request to the terminal, for example: "Please retransmit the original data D3 corresponding to hash value H3." Upon receiving the request, the terminal queries its local cache or uses the H3 index to package and retransmit the original data D3 separately. After receiving D3, cloud server 200 calculates its hash value and confirms that it matches the held, signed H3, thus completing the verification of D3 and closing the loop. If cloud server 200 detects a missing signature packet, it only requests a retransmission of the signature packet, without needing to resend the large original data packets 1 to n. In this invention, an asymmetric retransmission mechanism is adopted by decoupling the hash values ​​in the data packet and the signature packet. If the data packet is lost, only the original data needs to be retransmitted, without the need to retransmit other data and its signature. If the missing signature packet is lost, only the small signature packet needs to be retransmitted, without the need to retransmit the massive amount of original data. Compared with the full retransmission of "data + signature" in the traditional solution, in weak network environments such as narrowband IoT, invalid data transmission is greatly reduced, and the bandwidth consumption and terminal power consumption caused by retransmission are significantly reduced.

[0074] The following example illustrates the temperature monitoring of a high-voltage transmission line in a smart grid:

[0075] When a cable temperature sensor (terminal) is powered on, the microcontroller (MCU) control module monitors environmental parameters in real time, assesses that the current 4G network signal is good, the edge device battery is fully charged, the temperature data is normal data, and sets the batch signature threshold n=50.

[0076] The temperature sensor collects the temperature every second. If the current temperature is 25.1℃, it sends the data to the hardware security chip 3. After receiving the temperature data, the hardware security chip 3 calculates the SM3 hash value Hash1 using the built-in device private key and caches the hash value in its internal secure storage area. The encapsulation and transmission module 4 encapsulates the temperature data and the corresponding hash value into a data packet 1 and sends it to the cloud server 200. In this way, the encapsulation and transmission module 4 sends [data packet 1: temperature value + Hash1], [data packet 2: temperature value + Hash2], etc. to the cloud server 200 until the 50th second, when it sends [data packet 50: temperature value + Hash50].

[0077] After sending 50 data packets, the hardware security chip 3 uses the SM2 private key to uniformly sign the 50 hash values ​​accumulated internally to generate an overall digital signature, denoted as the SM2 signature. The encapsulation and transmission module 4 encapsulates the set of all hash values ​​and the overall digital signature into a signature packet [the set of 50 hashes + the SM2 signature], and sends it to the cloud server 200.

[0078] If the 30th data packet is lost due to storm interference, and the cloud server 200 receives 49 data packets and 1 signature packet, the cloud server 200 verifies the signature packet successfully, confirming the authenticity of this batch of data. The cloud server 200 checks the 30th hash value contained in the signature packet and sends a command to the terminal: "Retransmit only the 30th data packet." After the IoT terminal 100 retransmits the data, the closed loop is completed.

[0079] If three consecutive data packet transmission timeouts are detected or the signal strength is below the threshold, an adaptive adjustment mechanism is immediately triggered to dynamically lower the batch signature threshold n from 50 to 20 or lower, in order to shorten the single batch transmission cycle and reduce the risk of the entire batch of data being stuck due to network jitter.

[0080] If, at the 51st second, the sensor temperature suddenly rises to 120°C, suspected of a short circuit and fire, the control module 1 recognizes that the data value exceeds the safety threshold and becomes high-priority data. It immediately forces the batch signature threshold n to be set to 1, and the hardware security chip 3 immediately performs hash value calculation and signature for the alarm data separately, combining them into a high-priority emergency alarm packet [alarm data + hash + individual signature], which is sent by the encapsulation and transmission module 4 to ensure zero delay in disaster relief response.

[0081] This invention changes the traditional strongly bound encapsulation mode of "data + signature," separating the data packet and the signature packet into separate encapsulations. It changes the traditional "sign every packet must be signed" to "sign as you collect, batch signing," significantly reducing the frequency of asymmetric encryption by the hardware security chip, eliminating transmission blockage caused by encryption, and multiplying the system throughput. It breaks through the static limitations of fixed batch signing, dynamically adjusting the batch signing threshold according to environmental parameters. Without sacrificing data integrity and non-repudiation, it minimizes the signature calculation overhead of IoT devices and saves network bandwidth, achieving an adaptive optimal solution of "saving power and bandwidth for routine data, and ensuring real-time and security for emergency data." The entire process of hash value calculation, caching, and private key signing is completed within the physical security boundary of the hardware security chip. The device's private key is non-exportable from the date of its generation, and all cryptographic operations are performed in a closed loop within the chip. This protects against firmware reverse engineering, runtime memory theft, and physical side-channel attacks targeting the IoT main control MCU, ensuring the authenticity of the device's identity and the absolute security of the signature key. Utilizing a decoupled design between the data packet and the hash value in the signature packet, an asymmetric retransmission mechanism is adopted. If a data packet is lost, only the original data needs to be retransmitted, without retransmitting other data or its signature. If a missing signature packet is lost, only a smaller signature packet needs to be retransmitted, without retransmitting massive amounts of original data. Compared to the traditional solution of full retransmission of "data + signature", this significantly reduces invalid data transmission in weak network environments such as narrowband IoT, and significantly reduces bandwidth consumption and terminal power consumption caused by retransmission.

[0082] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for adaptive data collection and encapsulation in the Internet of Things (IoT), characterized in that, The method includes: The batch signature threshold is dynamically determined based on environmental parameters. Real-time data collection, hash value calculation and caching, and data and their corresponding hash values ​​are encapsulated into data packets and sent to the cloud; When the number of data packets sent reaches the batch signature threshold, a signature operation is triggered. The built-in private key is used to generate an overall digital signature for all cached hash values. The set of all hash values ​​and the overall digital signature are then packaged into a signature packet and sent to the cloud for cloud verification of data integrity. The overall data signature is verified using the public key to confirm the authenticity of the data packets in batches.

2. The IoT data adaptive collection and encapsulation method according to claim 1, characterized in that, The cloud-based verification of data integrity, and the batch confirmation of data packet authenticity through overall data signature, include: After receiving the data packet, the cloud recalculates the hash value and compares it with the hash value attached to the data packet to verify data integrity. After receiving the signature packet in the cloud, the cloud uses the public key to verify the overall digital signature. If the verification is successful, the authenticity of the data packets is confirmed in batches.

3. The IoT data adaptive collection and encapsulation method according to claim 1, characterized in that, The cloud also initiates retransmission requests based on the missing information: If a missing data packet is detected, based on the hash value stored in the signature packet, only the missing original data is requested to be retransmitted; If a missing signature packet is detected, only request a retransmission of the signature packet.

4. The IoT data adaptive acquisition and encapsulation method according to claim 1, characterized in that: The environmental parameters include one or more of the following: network quality, device power consumption, and data priority.

5. The IoT data adaptive acquisition and encapsulation method according to claim 3, characterized in that: When the network quality is better than the first preset threshold, the battery level is higher than the second preset threshold, and the data is of normal priority, the first batch signature threshold is set. When the network packet loss rate is higher than the third preset threshold or the signal strength is lower than the fourth threshold, the batch signature threshold is reduced to the second batch signature threshold, which is less than the first batch threshold and greater than 1. When alarm-type or high-priority burst data is detected, the batch signature threshold is forcibly set to 1.

6. The IoT data adaptive acquisition and encapsulation method according to claim 1, characterized in that, The process of generating an overall digital signature using the built-in private key includes: concatenating the hash values ​​corresponding to the collected data in the order in which the data packets were generated to form a sequence set, and signing the sequence set to generate an overall digital signature.

7. The IoT data adaptive acquisition and encapsulation method according to claim 1, characterized in that, The process of generating an overall digital signature using a built-in private key includes: constructing a hash tree, using the hash values ​​corresponding to the collected data as leaf nodes, pairing them up to calculate the upper-level hash values, recursively calculating layer by layer until the root hash value is calculated, and then signing the root hash value to generate an overall digital signature.

8. The IoT data adaptive acquisition and encapsulation method according to claim 1, characterized in that: After the signature packet is sent to the cloud, the cached hash value is cleared to prepare for the next signing cycle.

9. An adaptive data acquisition and packaging system for the Internet of Things, characterized in that: The system includes an Internet of Things (IoT) terminal and a cloud server; The IoT terminal includes: The control module is used to monitor environmental parameters in real time and dynamically determine the batch signature threshold based on the monitoring results. The data acquisition module is used to collect data in real time. The hardware security chip has a built-in, non-exportable device private key, which is used to calculate the hash value of the collected data and cache it in the internal secure storage area. When the batch signature threshold is reached, the device private key is used to generate an overall digital signature for all the cached hash values. The encapsulation and transmission module is used to encapsulate each piece of data and its corresponding hash value into a data packet and send it to the cloud server in real time. When the batch signature threshold is reached, the set of all hash values ​​and the overall digital signature are encapsulated into a signature packet and sent to the cloud server. The cloud server is used to receive data packets and signature packets, verify data integrity, verify the overall data signature using a public key, and confirm the authenticity of data packets in batches.

10. The IoT data adaptive acquisition and packaging system according to claim 9, characterized in that: The hardware security chip is a Security Element (SE) or a Trusted Platform Module (TPM), which integrates a cryptographic acceleration engine for high-speed hash calculations.

Images