A method and apparatus for storing evidence of internet of things data, a blockchain gateway device, and a computer program product

Abstract

The application belongs to the technical field of blockchains, and provides a method and device for storing evidence based on Internet of Things data, a blockchain gateway device and a computer program product, which comprise: receiving an original data message from an Internet of Things device, the original data message containing a device identifier, an original timestamp and a business load; performing standardized processing on the business load to generate a corresponding business load fingerprint; the standardized processing is used to eliminate format noise caused by serialization differences of the business load; then performing discretization processing on the original timestamp to obtain a corresponding discrete time slot starting point; then generating a globally unique logical data identifier based on the device identifier, the discrete time slot starting point and the business load fingerprint; finally, using a hierarchical progressive idempotent detection mechanism to determine the uniqueness of the logical data identifier, and if there is no historical record of the logical data identifier, performing a blockchain on-chain storage operation on the original data message to realize one thing one certificate and avoid repeated storage.

Description

Technical Field

[0001] This application belongs to the field of blockchain technology, and in particular relates to a method, device, blockchain gateway device and computer program product for evidence storage control based on Internet of Things data. Background Technology

[0002] In blockchain-based evidence storage applications for fields such as industrial IoT, smart cities, and supply chain traceability, to ensure the immutability and legal validity of massive amounts of time-series data, the industry generally adopts the approach of storing the hash value of the original data on the blockchain in technical practice.

[0003] However, current technical solutions face several key challenges in practical implementation, affecting the certainty, cost-effectiveness, and system stability of evidence preservation. First, the heterogeneous nature of IoT devices leads to non-deterministic differences in their serialized output of the same business semantic data. For example, inconsistent key-value pair order in JSON structures, inconsistencies in numerical precision and scientific notation, and different handling of whitespace can easily trigger a "hash avalanche" when directly calculating hashes on the original data. This means that semantically identical data generates completely different hash values ​​due to format differences, resulting in on-chain evidence preservation failing to anchor a unique business fact and significantly increasing the verification failure rate. Second, IoT devices are often deployed in environments with unstable network conditions. The retransmission mechanism of the underlying communication protocol can easily lead to duplicate data reporting. If the system lacks idempotency recognition capabilities, the same data will be uploaded to the chain multiple times, resulting in unnecessary resource consumption and cost waste in transaction-based blockchain networks. It also introduces redundant data on-chain, interfering with business status and auditing processes. Furthermore, clock skew between the device and server sides can cause subtle jitter in the timestamps corresponding to the same physical event. If the original timestamps are used directly to construct unique data identifiers, it will cause "primary key drift," meaning that the same event is identified by the system as multiple independent data entries due to slight differences in timestamps, thus disrupting the stability and continuity of the business primary key.

[0004] Therefore, these defects collectively affect the certainty and reliability of evidence storage in blockchain systems. Summary of the Invention

[0005] This application provides a method, apparatus, blockchain gateway device, and computer program product for evidence storage and control based on Internet of Things (IoT) data, which can solve the technical problems of insufficient determinism and reliability in existing blockchain evidence storage technologies mentioned in the background. This invention is applicable to Ethereum and heterogeneous blockchain architectures based on parachains, relay chains, etc., to achieve deterministic evidence storage and anti-replay control of cross-chain, multi-source IoT data.

[0006] In a first aspect, embodiments of this application provide a method for controlling the storage of IoT data. The method is applied to a blockchain gateway device connected to a blockchain network and includes the following steps:

[0007] S1: Receive raw data messages from IoT devices, the raw data messages including device identifier, raw timestamp and service load;

[0008] S2: Perform normalization processing on the service load to generate a corresponding service load fingerprint; the normalization processing is used to eliminate format noise caused by serialization differences in the service load;

[0009] S3: Based on a preset time window threshold Δ, perform discretization processing on the original timestamp to calculate the starting point of the discrete time slot corresponding to the original timestamp;

[0010] S4: Generate a globally unique logical data identifier based on the device identifier, the discrete time slot start point W, and the service load fingerprint;

[0011] S5: Use a hierarchical progressive idempotency detection mechanism to determine the uniqueness of the logical data identifier. If the determination result is that the logical data identifier has no historical record, perform a blockchain on-chain evidence storage operation on the original data message.

[0012] The technical effect of this application's embodiments is as follows: This application can establish a core process for deterministic evidence storage of IoT data. It generates stable logical identifiers by progressive semantic normalization and time discretization of the business load in the original data message. Finally, it uses a hierarchical progressive idempotent detection mechanism to determine the logical data identifiers to ensure their stability and uniqueness, thereby achieving one certificate for one event and avoiding duplicate evidence storage.

[0013] In one embodiment, the normalization process includes:

[0014] The nested data structure in the aforementioned workload is recursively flattened into n one-dimensional key-value pairs with complete paths;

[0015] Sort the key-value pairs according to the lexicographical order of their keys, where n is a positive integer;

[0016] Normalize the key values ​​in each sorted key-value pair.

[0017] The technical effect of this embodiment is that by forcing key name sorting and key value format normalization, it eliminates format noise caused by serialization differences, generates unique and deterministic digital fingerprints for business loads with the same semantics, and solves the hash avalanche problem from the root.

[0018] In one embodiment, the normalization process for the key values ​​in each sorted key-value pair includes:

[0019] For each key-value pair, remove any strings with trailing invalid zeros from the floating-point numbers in the key-value pairs.

[0020] Convert the boolean values ​​in each key-value pair into strings with consistent capitalization.

[0021] For each key-value pair, empty values ​​are converted into empty strings.

[0022] The technical effect of this embodiment is that it unifies the numerical format expression of different data sources, eliminates semantic ambiguity caused by system differences, and ensures that the same business data generates a consistent load fingerprint.

[0023] In one embodiment, the starting point of the discrete time slot is determined by the following formula:

[0024] W = t / Δ × Δ

[0025] Where t is the original timestamp, W is the starting point of the discrete time slot, and Δ is the time window threshold. This is a floor function used to round down to the nearest integer, representing the largest integer not greater than the value within the parentheses.

[0026] The technical advantage of this embodiment is that it maps continuous timestamps to discrete time slots, tolerates device clock deviations and network latency, and solves the primary key drift problem caused by timestamp jitter.

[0027] In one embodiment, the hierarchical progressive idempotent detection mechanism includes a memory-level probabilistic interception operation and a persistent interception operation;

[0028] Accordingly, the memory-level probabilistic interception operation includes the following steps:

[0029] A filter constructed based on a probabilistic data structure is used to determine whether the logical data identifier already exists.

[0030] If the filter determines that the logical data identifier does not exist, then the logical data identifier does not have a historical record, and the original data message is stored on the blockchain.

[0031] Using a probability filter to predict duplicate data at the millisecond level, the system blocks the vast majority of short-term retransmission requests, significantly reducing the load on the backend validation process.

[0032] Accordingly, the persistent interception operation includes the following steps:

[0033] If the logical data identifier is determined to exist based on the filter, the record containing the logical data identifier is inserted into the database with unique index constraints. If the logical data identifier is successfully inserted into the database, it is determined that the logical data identifier has no historical records, and the blockchain on-chain evidence storage operation is performed on the original data message.

[0034] The technical advantages of this embodiment are as follows: the memory-level probabilistic interception operation uses a probabilistic filter to achieve millisecond-level prediction of duplicate data, intercepting most short-term retransmission requests and significantly reducing the backend verification load; while the persistent interception operation can be persistently verified through a local relational database, solving the misjudgment problem of the memory-level probabilistic interception operation and ensuring that the system can still maintain idempotency after restarting or an anomaly.

[0035] In one embodiment, during the persistence interception operation, if the logical data identifier insertion fails due to a conflict with the unique index, it is determined that the logical data identifier has a historical record, and the historical blockchain transaction hash corresponding to the logical data identifier is retrieved from the database and returned to intercept the initiation of a new blockchain transaction operation.

[0036] The technical advantage of this embodiment is that it uses a unique index in a local relational database to achieve accurate deduplication, directly returns historical evidence records, and completely avoids the ineffective consumption of blockchain network resources.

[0037] In one embodiment, the service load fingerprint is generated in the following manner:

[0038] The data obtained after normalization is hashed using either the Chinese national cryptographic SM3 algorithm or the international standard SHA-256 algorithm.

[0039] The logical data identifier is generated in the following way: using the national cryptographic SM3 algorithm or the international standard SHA-256 algorithm to perform hash calculation on the device identifier, the discrete time slot start point and the service load fingerprint.

[0040] The technical advantage of this embodiment is that it supports configurable switching between Chinese national cryptographic algorithms and international cryptographic algorithms, meeting different security compliance requirements, while maintaining the consistency of the fingerprint generation mechanism.

[0041] Secondly, embodiments of this application also provide an IoT data storage and control device, the device being deployed on a blockchain gateway device connected to a blockchain network, the device comprising:

[0042] The receiving module is used to receive raw data messages from IoT devices, the raw data messages including device identifier, raw timestamp and service load;

[0043] A normalization processing module is used to perform normalization processing on the business load and generate a corresponding business load fingerprint; the normalization processing is used to eliminate format noise caused by serialization differences in the business load;

[0044] The time window discretization processing module is used to perform discretization processing on the original timestamp based on a preset time window threshold, and calculate the starting point of the discrete time slot corresponding to the original timestamp.

[0045] The logical identifier generation module is used to generate a globally unique logical data identifier based on the device identifier, the discrete time slot start point, and the service load fingerprint;

[0046] The detection and transaction module is used to determine the uniqueness of the logical data identifier using a hierarchical progressive idempotent detection mechanism. If the determination result is that the logical data identifier has no historical record, the module performs a blockchain-based on-chain evidence storage operation on the original data message.

[0047] Thirdly, embodiments of this application also provide a blockchain gateway device, including: a processor and a memory, the processor being connected to the memory, the memory being used to store a computer program, and the processor being used to execute the computer program stored in the memory, so that the blockchain gateway device performs the evidence storage control method based on Internet of Things data as described in the first aspect.

[0048] Fourthly, embodiments of this application also provide a computer program product, including an evidence storage control execution program based on Internet of Things (IoT) data. When the IoT data-based evidence storage control execution program is run by a blockchain gateway device, the steps of the IoT data-based evidence storage control method described in the first aspect are executed.

[0049] It should be noted that the technical effects of the second and fourth aspects mentioned above are basically the same as those of the first aspect mentioned above, and will not be described in detail in the embodiments of this application. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 The main flowchart of the evidence storage control method based on Internet of Things data provided in this application;

[0052] Figure 2A flowchart of an embodiment of the evidence storage control method based on Internet of Things data provided in this application;

[0053] Figure 3 A structural block diagram of the evidence storage control device based on Internet of Things data provided in this application;

[0054] Figure 4 This is a schematic diagram of the structure of a blockchain gateway device according to an embodiment of this application. Detailed Implementation

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0056] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0057] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0058] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), unless otherwise expressly and specifically defined.

[0059] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0060] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0061] Understandably, in scenarios such as the Industrial Internet of Things (IIoT), smart cities, and supply chain traceability, massive amounts of sensor devices continuously generate time-series data. To ensure the immutability and legal validity of the data, storing the data on the blockchain has become an industry consensus. A common technical approach is to calculate a hash value for the original data and then write the hash value (rather than the original data) into a blockchain smart contract.

[0062] Through in-depth analysis of existing technical solutions, the applicant of this invention has identified the following three core technical pain points:

[0063] Defect 1: "Hash avalanche" caused by the nondeterminism of serialization

[0064] Phenomenon Description: IoT devices from different manufacturers and with different firmware versions exhibit differences in their JSON serialization output of the same business data. Typical differences include:

[0065] The order of key-value pairs is different (e.g., {"a":1, "b":2} vs {"b":2, "a":1}).

[0066] The numerical precision is expressed differently (e.g., 25 vs 25.0 vs 2.5e1).

[0067] Whitespace and newline characters are handled differently.

[0068] Technical consequences: Directly calculating the hash (such as SHA256) of the original payload can trigger an "avalanche effect"—two pieces of data with completely identical business semantics can produce drastically different hash values ​​due to format differences. This makes it impossible to anchor a unique business fact on the blockchain, resulting in a failure rate of over 30% for evidence storage and verification.

[0069] Defect 2: "Double consumption" caused by retransmission in weak networks

[0070] Phenomenon Description: IoT devices are typically deployed in weak network environments such as factory workshops and outdoor base stations, making them highly susceptible to message retransmission mechanisms, such as MQTT / CoAP. The traditional "receive and connect immediately" strategy cannot distinguish between new data and retransmitted data.

[0071] Technical Consequences: In transaction-based blockchain networks (such as Ethereum, FISCO, and BCOS), each transaction consumes Gas. Repeated on-chain operations lead to ineffective waste of Gas fees; actual tests show that abnormal costs can reach 15%-30% of normal costs. More seriously, redundant garbage data is generated on-chain, interfering with the state logic of smart contracts and subsequent auditing.

[0072] Defect 3: "Primary key drift" caused by timestamp jitter:

[0073] Phenomenon description: There is often a subtle deviation (usually hundreds of milliseconds to several seconds) between the local clock of the IoT device and the clock of the gateway server, which causes the same event to appear with different timestamps on different processing nodes.

[0074] Technical consequences: If the original timestamp is directly included in the calculation factor of the data unique identifier, the same physical event will generate multiple different logical IDs, making it impossible to form a stable business primary key.

[0075] In summary, existing evidence storage schemes have systemic defects in data standardization, retransmission deduplication, and time synchronization. To address these defects, this application provides an evidence storage control method, a blockchain gateway device, and a computer program product based on Internet of Things (IoT) data. This achieves a deterministic evidence storage scheme that can unify data fingerprints at the semantic level, resist duplicate network submissions, and tolerate clock jitter. The technical solution of this application will be illustrated below through specific embodiments.

[0076] Firstly, such as Figure 1 As shown, this application provides a method for controlling the storage of IoT data, applied to a blockchain gateway device connected to a blockchain network (i.e., the blockchain gateway device receives data messages, processes data messages, and stores the data messages on the blockchain as evidence), mainly including the following steps S1 to S5:

[0077] Step S1: Receive raw data messages from IoT devices, the raw data messages including device identifier, raw timestamp and service load;

[0078] Step S2: Perform normalization processing on the service load to generate a corresponding service load fingerprint; the normalization processing is used to eliminate format noise caused by serialization differences in the service load;

[0079] Step S3: Based on a preset time window threshold Δ, perform discretization processing on the original timestamp to calculate the starting point of the discrete time slot corresponding to the original timestamp;

[0080] Step S4: Generate a globally unique logical data identifier based on the device identifier, the discrete time slot start point W, and the service load fingerprint;

[0081] Step S5: Use a hierarchical progressive idempotency detection mechanism to determine the uniqueness of the logical data identifier. If the determination result is that the logical data identifier has no historical record, perform a blockchain on-chain evidence storage operation on the original data message.

[0082] This application embodiment can establish a core process for deterministic evidence storage of IoT data. It generates stable logical identifiers by progressive semantic normalization and time discretization of the business load in the original data message. Finally, it uses a hierarchical progressive idempotent detection mechanism to determine the logical data identifiers to ensure their stability and uniqueness, thereby achieving one certificate per event and avoiding duplicate evidence storage.

[0083] It is understood that the IoT data storage control method of this application embodiment can be applied to weak network environments such as factory workshops and field base stations. The main body of this solution is the blockchain gateway device that accesses the blockchain network. The blockchain gateway device can be an edge gateway in the IoT data acquisition and preprocessing layer. It is usually deployed between the device and the blockchain network and is responsible for receiving data from multi-source heterogeneous IoT devices and performing semantic normalization, idempotent control and other processing.

[0084] For example, a blockchain gateway device (hereinafter referred to as the gateway device) can be an access node or client of a blockchain network, responsible for submitting the processed data fingerprint (i.e., the aforementioned generated logical data identifier) ​​to the blockchain network for notarization. The gateway device does not directly write the original data, but instead writes the deterministic fingerprint of the data into the blockchain to ensure the uniqueness and immutability of the data.

[0085] In some embodiments, for the aforementioned step S1, the original data message mainly comes from IoT devices. For example, IoT devices can be sensors, smart terminals, and other devices used in scenarios such as factory workshops and field base stations.

[0086] For example, taking the original message M_raw as an example, the IoT device can send the original message M_raw to the blockchain gateway device in the form of JSON or a nested Map structure via protocols such as MQTT and CoAP;

[0087] It should be noted that the "raw data message" "received" in step S1 usually originates from a specific IoT device, meaning the blockchain gateway device is processing a single message at this point. Steps S1 to S5 of the IoT data storage and control method provided by this invention are a general processing method. The blockchain gateway device will continuously run this method, therefore steps S1 to S5 will be executed repeatedly to process messages from numerous IoT devices within the network.

[0088] In this embodiment, the raw data message M_raw includes a device identifier, a raw timestamp, and a service load; the device identifier, raw timestamp, and service load are associated data within a single raw data message. A single raw data message contains only one device identifier. It can be understood that in IoT protocols such as MQTT and CoAP, a message (or a data packet) typically has a clear single source. The device identifier (such as DeviceID) is metadata used to identify the data sender; therefore, the correspondence between the device identifier, raw timestamp, and service load is as follows: within a message, one device identifier corresponds to one (or a set of associated) raw timestamps, and one service load generated at that moment; together, they constitute an indivisible data unit.

[0089] Further, refer to Figure 2 In one specific embodiment, the normalization process of step S2 described above can be performed by executing the following sub-steps S1.1 to S1.4:

[0090] Sub-step S1.1: Recursively flatten the nested data structure in the business load into n one-dimensional key-value pairs with complete paths;

[0091] For example, the workload is structured measurement data collected by the IoT sensor, wherein the structured measurement data is in JSON format or a key-value pair mapping. For instance, a blockchain gateway device receives a raw message M_raw in any format (the raw message M_raw can be in JSON format or a nested Map structure).

[0092] It's understandable that a raw data message contains one and only one payload, but this payload itself might be a JSON object containing n key-value pairs. Take the following raw message M_raw code as an example:

[0093] A raw data message M_raw (e.g., a JSON message)

[0094] {

[0095] "deviceId": "sensor_001", / / Device identifier (metadata)

[0096] "timestamp": 1640995200000, / / Timestamp (metadata)

[0097] "payload": { / / Business payload (only one, but it is structured)

[0098] "lat": 39.9, / / Key-value pair 1

[0099] "lng": 116.4 / / 2 key-value pairs

[0100] "sensor": { / / Nested structure, containing more key-value pairs

[0101] "temperature": 25.5, / / Key-value pair 3

[0102] "humidity": 60, / / 4 key-value pairs

[0103] }

[0104] }

[0105] }

[0106] As can be seen from the code example above in the embodiments of this application, the business load includes a nested data structure; in a specific implementation, the gateway's processor can traverse the JSON / Map tree structure of the original message M_raw and expand the nested structure into a one-dimensional key-value pair; in this embodiment, the key name Key is named by concatenating the complete path using the dot-matrix method;

[0107] As an example, the dot-matrix method in this embodiment is a way to concatenate nested JSON paths into flattened key names. For instance, for the original message M_raw with the nested structure {"sensor": {"temp": 25.0, "humidity": 60}}, the complete path of the key name "temp" would be concatenated to "sensor.temp". This method preserves the hierarchical semantics of the data, facilitating subsequent lexicographical sorting and hash calculations, for example:

[0108] Example input: {"sensor": {"temp": 25.0, "humidity": 60}};

[0109] Example output: [("sensor.humidity", "60"), ("sensor.temp", "25")].

[0110] Sub-step S1.2: Sort the key-value pairs according to the lexicographical order of their key names:

[0111] It's understandable that the order of keys appearing in the raw data messages of a conventional solution may be uncertain, potentially varying due to differences in device manufacturers, serialization libraries, programming languages, or transmission protocols. For example, the same data might appear in the form of {"a":1, "b":2} or {"b":2, "a":1}. Another example: suppose the same temperature sensor in a conventional solution generates the same business data at the same time. However, because vendor A and vendor B use different JSON libraries, the order of keys appearing in their payloads might be different:

[0112] Payload sent by device A from manufacturer: {"temperature": 25.0, "humidity": 60};

[0113] Payload sent by Manufacturer B's device: {"humidity": 60, "temperature": 25};

[0114] If you concatenate strings directly without sorting to calculate the hash: A's concatenated string is "temperature=25&humidity=60"; B's concatenated string is "humidity=60&temperature=25". Although their business semantics (temperature 25, humidity 60) are exactly the same, the hashes calculated by concatenating strings directly without sorting are completely different.

[0115] To address the above issues, as an example, in a specific implementation, this application embodiment ignores the order in which the Keys appear in the original data message and forces all key-value pairs after flattening in sub-step S1.1 to be sorted in ascending order according to the lexicographical order of the Keys' ASCII codes. This step eliminates the problem of different key orders caused by implementation differences among different JSON serialization libraries.

[0116] It is understood that after sorting by applying sub-step S1.2 of the embodiment of this application: regardless of the original order, it is forced to be sorted in ascending order according to the ASCII lexicographical order of the Key, with "humidity" before "temperature";

[0117] After sorting, the flattened key-value pairs of A and B become: [("humidity", "60"), ("temperature", "25")];

[0118] The standardized concatenation string is: "humidity=60&temperature=25";

[0119] Calculated business load fingerprint Hpayload Totally consistent.

[0120] This achieves "semantic determinism"—as long as the objective facts expressed by the data are the same, regardless of the differences in their serialization formats, the same unique fingerprint will eventually be generated, and the determinism of device identifiers and timestamps will be guaranteed by subsequent steps.

[0121] Sub-step S1.3: Normalize the key values ​​in each sorted key-value pair;

[0122] As an example, the key-value pairs in the sorted key-value pairs can be normalized as follows:

[0123] For each key-value pair, remove the string format of the floating-point number in the key value; for example, for the floating-point number in the key value: uniformly convert it to the string format of removing the trailing invalid zeros; 25.00 → "25", 0.100 → "0.1", 1.0e2 → "100"; to eliminate the differences in floating-point number representation under different computer architectures;

[0124] For each key-value pair, the boolean value in the key is converted into a string with consistent uppercase and lowercase format. For example, for the boolean value in the key: all are converted into strings with consistent uppercase and lowercase format, such as converting them to lowercase strings: True → "true", FALSE → "false", in order to eliminate the case difference of boolean values ​​under different computer architectures.

[0125] For each key-value pair, empty values ​​are converted into empty strings; for example, empty values ​​in the key-value pairs are uniformly converted into empty strings "" to eliminate the differences in the representation of empty values ​​under different computer architectures.

[0126] Sub-step S1.4: Based on the target key-value pair data that has been sorted by key name and normalized by key value, generate the corresponding business load fingerprint;

[0127] For example, in this embodiment, key-value pairs (Key, Value) sorted by key name and normalized by value can be concatenated into a normalized string according to a fixed format, such as "k1=v1&k2=v2&..." (without a separator at the end), and then its cryptographic hash value can be calculated and labeled as H. payload .

[0128] In the specific implementation, the target key-value pair data (after key name sorting and key value normalization) obtained through the aforementioned sub-steps S1.1 to S1.4 can be denoted as CanonicalString(Mraw). The cryptographic hash value H of the target key-value pair data is calculated using the Chinese national cryptographic SM3 algorithm and the international standard SHA-256 algorithm. payload H payload = HashFunc(CanonicalString(Mraw));

[0129] HashFunc() is a cryptographic hash function that supports configurable switching between the SHA-256 (international standard) algorithm and the SM3 (national cryptographic standard) algorithm.

[0130] The final output service load fingerprint H payload It can be a fixed-length hexadecimal string (e.g., the result of SHA-256 is a 64-bit hexadecimal number), which can be a unique, compact, and tamper-resistant digital digest of the original business data M_raw at the semantic level.

[0131] The technical effect of this embodiment is that by forcing key name sorting and key value format normalization, it eliminates format noise caused by serialization differences, generates unique and deterministic digital fingerprints for business loads with the same semantics, and solves the hash avalanche problem from the root.

[0132] Furthermore, in this embodiment, for the aforementioned step S3, this application embodiment changes the "continuous precise timestamp" to a "discrete time slot" to solve the "primary key drift" caused by the aforementioned timestamp jitter. It can be understood that the cause of the aforementioned timestamp jitter may be that the clock on the IoT device (t_device) is not synchronized with the clock on the blockchain gateway device (t_gateway), resulting in a fixed or drifting deviation δ. For example, for the same physical event, the IoT device records the time as t1, while the gateway may record the time as t1' (if the gateway re-stamps) or directly use t1. In any case, the "timestamp factor" used to calculate the identifier is uncertain. If this unstable timestamp is directly included in the hash calculation: ID = Hash(DeviceID || t || Payload), then a small change in t will cause the hash value to "avalanche," changing the entire ID accordingly; this is "primary key drift."

[0133] To address the aforementioned issues, this embodiment performs time window discretization processing on the original timestamps obtained from the acquisition in S1, mapping them to discretized time slots:

[0134] For example, this embodiment uses the following formula:

[0135] W = t / Δ × Δ

[0136] Where t is the original timestamp (accurate to milliseconds), W is the starting point of the discrete time slot, and Δ is the time window threshold. This is the floor function, used to denote the largest integer not greater than the value within the parentheses;

[0137] Understandably, the time window discretization processing in this application embodiment can divide a continuous time axis t (i.e., the original timestamp) into continuous, non-overlapping time slots using a fixed time window threshold Δ as the unit. Any original timestamp falling within the same time slot [nΔ, (n+1)Δ) will output the same discrete value nΔ after being processed by the function W.

[0138] As an example, suppose the physical event occurs at real time T;

[0139] The original timestamp recorded by device A: t_a = T + δ_a (δ_a is the clock offset of A);

[0140] The original timestamp recorded by device B (or different reporting paths of the same device): t_b = T + δ_b;

[0141] The difference between the two (i.e. the jitter) is: |t_a - t_b| = |δ_a - δ_b|.

[0142] If the absolute value of this jitter is less than the time window threshold Δ, i.e. |δ_a - δ_b| < Δ, then it can be proven through mathematical derivation that t_a and t_b have a high probability of being mapped to the same integer multiple nΔ.

[0143] In some embodiments, a time window threshold Δ (e.g., 5 seconds, 10 seconds) much larger than |δ_a - δ_b| can be set based on the maximum network latency and the maximum expected clock deviation. In this way, as long as the device clock deviation and network latency are within the expected range, all original timestamps generated for the same event will be forced to be equal after processing by the W(t, Δ) function.

[0144] Furthermore, in this embodiment of the application, for the aforementioned step S4, the originally unstable variable t is replaced with a stable, discretized value W(t, Δ);

[0145] As an example, this application embodiment uses the national cryptographic SM3 algorithm or the international standard SHA-256 algorithm, based on the device identifier DeviceID, the discrete time slot start point W(t, Δ), and the service load fingerprint H. payload Generate a globally unique logical data identifier ID. logic = HashFunc(DeviceID || W(t, Δ) ||H payload );

[0146] It's understandable that the calculation formula for the primary key identifier changes from the volatile Hash(DeviceID || t || Payload) to a stable ID. logic = HashFunc(DeviceID || W(t, Δ) || H payload At this point, the slight jitter of the original timestamp is "blurred" out and no longer affects the generation of the primary key. For the same physical event, no matter how slightly the recorded timestamp changes, as long as it falls within the same designed time slot, a globally unique and stable logical data identifier will be generated, and the primary key will no longer "drift". This allows multiple retransmissions by IoT devices within the Δ time window (even if the local timestamp changes slightly due to network latency), as long as the business content H... payload The generated logical data identifier ID remains unchanged. logic It remains constant and unique, fundamentally solving the problem of timestamp jitter.

[0147] Furthermore, in this embodiment of the application, regarding the aforementioned step S5, before initiating the blockchain on-chain transaction action on the original data message, the logical data identifier ID... logic The following hierarchical progressive idempotency detection mechanism can be executed (the hierarchical progressive idempotency detection mechanism includes a first-level memory-level probabilistic interception operation and a second-level persistent interception operation):

[0148] The first-level memory-level probabilistic interception operation: a filter constructed based on a probabilistic data structure determines whether the logical data identifier already exists;

[0149] As an example, this application embodiment uses a Bloom filter or a Cuckoo filter to probabilistically determine the existence of logical data identifiers; the detection effect of this embodiment can achieve an interception success rate of over 99%, and can realize short-term high-frequency repeated requests, with a query latency of less than 1ms, and extremely low memory usage (for example, only about 120MB is needed for 100 million records).

[0150] In a specific implementation, this application embodiment can query a Bloom filter or a Cuckoo filter deployed in the Redis cache database:

[0151] If the query result indicates that the logical data identifier ID does not exist. logic If the logical data identifier does not have a historical record, then the subsequent blockchain on-chain evidence storage operation in step S5 can continue.

[0152] If the query result indicates that the logical data identifier ID exists. logic If this is not the case, it indicates that the logical data identifier may have a historical record, and then the process proceeds to the second-level persistent interception layer for precise verification.

[0153] The second layer of persistent interception operation: insert the record containing the logical data identifier into a local relational database with unique index constraints, and determine whether the insertion into the database can be successful;

[0154] As an example, the local relational database in this embodiment can be deployed locally on the "blockchain gateway device" or on a dedicated server located on the same trusted intranet as it;

[0155] In this embodiment, we can attempt to insert evidence records into a local relational database (such as MySQL) and verify them using the UK_DataID unique index constraint.

[0156] If the logical data identifier is successfully inserted into the database, it indicates that the logical data identifier has no historical record, and the subsequent blockchain on-chain evidence storage operation in step S5 can continue to be executed.

[0157] If the insertion fails due to a unique index conflict, it is determined that the logical data identifier has a historical record. The database is then searched for and the historical on-chain transaction hash (TxHash) corresponding to the logical data identifier is returned directly to the caller to intercept the initiation of a new blockchain transaction operation and avoid duplicate on-chain operations. This ensures idempotency after a node crash and restart, and resolves the issue of missing accurate verification results due to probabilistic misjudgments in the first-level memory interception layer.

[0158] Furthermore, in other embodiments, if the application scenario of the blockchain network involves local database loss in extreme cases, such as when switching back and forth between the master and slave databases, the embodiments of this application can, in addition to ensuring the first-level memory interception layer and the second-level persistence interception layer, also call the read-only view function of the blockchain system's smart contract to identify the logical data ID. logic Make further judgments:

[0159] If the execution feedback result of the read-only view function is a read failure, it can be finally ensured that the logical data identifier has no historical records, and the subsequent blockchain on-chain evidence storage operation in step S5 can continue to be executed.

[0160] If the execution feedback result of the read-only view function is that the read was successful, it is determined that the logical data identifier has a historical record, the log is recorded and an alarm prompt is generated.

[0161] This application embodiment identifies the logical data by calling a read-only view function of a smart contract. logic Further assessment suggests that for high-value assets (such as large transaction certificates and key evidence), a final "double insurance" can be provided to prevent on-chain double-spending on the blockchain network due to extreme circumstances (such as data loss during master-slave switching) in the local database.

[0162] On the other hand, such as Figure 3 As shown, the present invention also provides an IoT data storage and control device, the device being deployed on a blockchain gateway device connected to a blockchain network, the device comprising:

[0163] The receiving module 10 is used to receive raw data messages from IoT devices, the raw data messages including device identifier, raw timestamp and service load;

[0164] The normalization processing module 20 is used to recursively flatten the nested data structure in the business load into n one-dimensional key-value pairs with complete paths, sort each key-value pair according to the lexicographical order of the key names of each key-value pair, where n is a positive integer; and normalize the key values ​​in each sorted key-value pair; and generate the corresponding business load fingerprint based on the target key-value pair data after key name sorting and key value normalization.

[0165] The time window discretization processing module 30 is used to perform discretization processing on the original timestamp based on a preset time window threshold, and calculate the starting point of the discrete time slot corresponding to the original timestamp.

[0166] The logical identifier generation module 40 is used to generate a globally unique logical data identifier based on the device identifier, the discrete time slot start point, and the service load fingerprint.

[0167] The detection and transaction module 50 is used to determine the uniqueness of the logical data identifier using a hierarchical progressive idempotent detection mechanism. If the determination result is that the logical data identifier has no historical record, the module performs a blockchain on-chain evidence storage operation on the original data message.

[0168] It should be noted that the information interaction and execution process between the smart contracts of the above modules / operation terminals / blockchain network are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0169] Furthermore, this application also proposes a blockchain gateway device. Figure 4 This is a schematic diagram of the structure of a blockchain gateway device provided in one embodiment of this application. Figure 4 As shown, the blockchain gateway device 7 of this embodiment includes: at least one processor 70 ( Figure 4 (Only one is shown in the image) a processor, a memory 71, and a computer program 72 stored in the memory 71 and executable on the at least one processor 70, wherein the processor 70 executes the computer program 72 to implement the steps in any of the above embodiments of the evidence storage control method based on Internet of Things data.

[0170] It is understood that the blockchain gateway device in this embodiment is connected to the blockchain network. The blockchain gateway device can be an edge gateway in the IoT data acquisition and preprocessing layer. It is usually deployed between the device and the blockchain network and is responsible for receiving data from multi-source heterogeneous IoT devices and performing semantic normalization, idempotent control and other processing.

[0171] For example, a blockchain gateway device (hereinafter referred to as the gateway device) can be an access node or client of a blockchain network, responsible for submitting the processed data fingerprint (i.e., the aforementioned generated logical data identifier) ​​to the blockchain network for notarization. The gateway device does not directly write the original data, but instead writes the deterministic fingerprint of the data into the blockchain to ensure the uniqueness and immutability of the data.

[0172] In one embodiment, the blockchain gateway device can be an electronic device such as a tablet computer, laptop, or PDA. The blockchain gateway device may include, but is not limited to, a processor 70 and a memory 71. Those skilled in the art will understand that... Figure 4 This is merely an example of a blockchain gateway device and does not constitute a limitation on blockchain gateway devices. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components, such as input / output devices, network access devices, etc.

[0173] The processor 70 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0174] In some embodiments, the memory 71 may be an internal storage unit of the blockchain gateway device, such as a hard drive or memory of the blockchain gateway device. In other embodiments, the memory 71 may be an external storage device of the blockchain gateway device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the blockchain gateway device. Furthermore, the memory 71 may include both internal and external storage units of the blockchain gateway device. The memory 71 is used to store the operating system, applications, bootloader, data, and other programs, such as the program code of the computer program. The memory 71 can also be used to temporarily store data that has been output or will be output.

[0175] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0176] This application also provides a computer program product, including an evidence storage control execution program based on Internet of Things (IoT) data. When the IoT data-based evidence storage control execution program is run, the steps of the IoT data-based evidence storage control method described in the first aspect are executed.

[0177] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0178] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

[0179] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0180] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0181] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0182] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0183] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for controlling the storage of Internet of Things (IoT) data, characterized in that, The method is applied to blockchain gateway devices accessing a blockchain network and includes the following steps: Receive raw data messages from IoT devices, the raw data messages including device identifier, raw timestamp and service load; The business load is normalized to generate a corresponding business load fingerprint; the normalization process is used to eliminate format noise caused by serialization differences in the business load. Based on a preset time window threshold, the original timestamp is discretized to calculate the starting point of the discrete time slot corresponding to the original timestamp. Based on the device identifier, the discrete time slot start point, and the service load fingerprint, a globally unique logical data identifier is generated; The logical data identifier is uniquely determined by a hierarchical progressive idempotent detection mechanism. If the determination result is that the logical data identifier has no historical record, the original data message is stored on the blockchain. The hierarchical progressive idempotent detection mechanism includes memory-level probabilistic interception operations and persistent interception operations: The memory-level probabilistic interception operation includes the following steps: A filter constructed based on a probabilistic data structure is used to determine whether the logical data identifier already exists. If the filter determines that the logical data identifier does not exist, then the logical data identifier does not have a historical record, and a blockchain on-chain evidence storage operation is performed on the original data message; The persistent interception operation includes the following steps: If the logical data identifier is determined to exist based on the filter, the record containing the logical data identifier is inserted into the database with unique index constraints. If the logical data identifier is successfully inserted into the database, it is determined that the logical data identifier has no historical records, and the blockchain on-chain evidence storage operation is performed on the original data message.

2. The method as described in claim 1, characterized in that, The normalization process includes: The nested data structure in the aforementioned workload is recursively flattened into n one-dimensional key-value pairs with complete paths, where n is a positive integer; Sort the key-value pairs according to the lexicographical order of their key names; Normalize the key values ​​in each sorted key-value pair.

3. The method as described in claim 2, characterized in that, The normalization process for the key values ​​in each sorted key-value pair includes: For each key-value pair, remove any strings with trailing invalid zeros from the floating-point numbers in the key-value pairs. Convert the boolean values ​​in each key-value pair into strings with consistent capitalization. For each key-value pair, empty values ​​are converted into empty strings.

4. The method according to any one of claims 1 to 3, characterized in that, The starting point of the discrete time slot is determined by the following formula: W = t / D × D Where t is the original timestamp, W is the starting point of the discrete time slot, and Δ is the time window threshold. This is a floor function used to round down to the nearest integer, representing the largest integer not greater than the value within the parentheses.

5. The method as described in claim 1, characterized in that, In the persistence interception operation, if the logical data identifier insertion fails due to a conflict in the unique index, it is determined that the logical data identifier has a historical record, and the historical blockchain transaction hash corresponding to the logical data identifier is searched from the database and returned to intercept the initiation of a new blockchain transaction operation.

6. The method according to any one of claims 1 to 3, characterized in that, The service load fingerprint is generated in the following ways: The data obtained after normalization is hashed using either the Chinese national cryptographic SM3 algorithm or the international standard SHA-256 algorithm. The logical data identifier is generated in the following way: using the national cryptographic SM3 algorithm or the international standard SHA-256 algorithm to perform hash calculation on the device identifier, the discrete time slot start point and the service load fingerprint.

7. A device for storing and controlling Internet of Things (IoT) data, characterized in that, The device is deployed on a blockchain gateway device connected to a blockchain network, and the device includes: The receiving module is used to receive raw data messages from IoT devices, the raw data messages including device identifier, raw timestamp and service load; A normalization processing module is used to perform normalization processing on the business load and generate a corresponding business load fingerprint; the normalization processing is used to eliminate format noise caused by serialization differences in the business load; The time window discretization processing module is used to perform discretization processing on the original timestamp based on a preset time window threshold, and calculate the starting point of the discrete time slot corresponding to the original timestamp. The logical identifier generation module is used to generate a globally unique logical data identifier based on the device identifier, the discrete time slot start point, and the service load fingerprint; The detection and transaction module is used to determine the uniqueness of the logical data identifier using a hierarchical progressive idempotent detection mechanism. If the determination result is that the logical data identifier has no historical record, the module performs a blockchain on-chain evidence storage operation on the original data message. The hierarchical progressive idempotent detection mechanism includes memory-level probabilistic interception operations and persistent interception operations: The memory-level probabilistic interception operation includes: A filter constructed based on a probabilistic data structure is used to determine whether the logical data identifier already exists. If the filter determines that the logical data identifier does not exist, then the logical data identifier does not have a historical record, and a blockchain on-chain evidence storage operation is performed on the original data message; The persistent interception operation includes: If the logical data identifier is determined to exist based on the filter, the record containing the logical data identifier is inserted into the database with unique index constraints. If the logical data identifier is successfully inserted into the database, it is determined that the logical data identifier has no historical records, and the blockchain on-chain evidence storage operation is performed on the original data message.

8. A blockchain gateway device, characterized in that, include: A processor and a memory, the processor being connected to the memory, the memory being used to store a computer program, the processor being used to execute the computer program stored in the memory, so that the blockchain gateway device performs the IoT data storage control method as described in any one of claims 1-6.

9. A computer program product, comprising an execution program for storing and controlling Internet of Things (IoT) data, characterized in that, When the IoT data storage control execution program is run by the blockchain gateway device, the steps of the IoT data storage control method as described in any one of claims 1 to 6 are executed.

Images