Scientific research instrument-oriented flow data block level evidence chain encryption storage system and method

Abstract

This invention provides a streaming data block-level evidence chain encryption and storage system and method for scientific research instruments, comprising: a scientific data output module for outputting byte stream data; an encryption device for performing the following operations: streaming the byte stream data into multiple data blocks; calculating the hash value of each data block and constructing a block-level evidence chain; encrypting to generate ciphertext blocks; generating a list for signing; generating a block fingerprint set digest; and a data management module for performing the following operations: performing replay attack and / or data gap detection on the session, and after passing the detection, constructing a Merkle tree from the hash values ​​of the data blocks; dynamically selecting the anchoring period based on traceability delay constraints and cost constraints; writing the root hash of the Merkle tree and the list signature as evidence records into immutable storage; and generating and returning a confirmation code to the encryption device. The beneficial effects of this invention are that it achieves zero-plaintext storage, block-level verifiability, low-cost anchoring, adaptive adjustment, and secure retransmission during network outages.

Description

Technical Field

[0001] This invention relates to the technical field of data encryption and evidence preservation, and more specifically, to a streaming data block-level evidence chain encryption and evidence preservation system and method for scientific research instruments. Background Technology

[0002] With the increasing digitization of scientific instruments such as microscopic imaging, gene sequencing, and high-energy physics detection, scientific data exhibits significant characteristics of high throughput, continuous streaming, and strong real-time performance. For example, raw data generated by devices such as cryo-electron microscopes and high-throughput sequencers can reach rates of hundreds of MB / s or even GB / s. This type of raw scientific data has extremely high value density and is usually impossible to re-collect, requiring that it possess the characteristics of immutability, verifiability, and auditability throughout its entire lifecycle of generation, transmission, storage, and traceability.

[0003] Currently, the following technical solutions are mainly used for secure data storage: Transport Layer Security (TLS / SSL) is used to protect the confidentiality of data transmission. File-level encryption (such as RSA / AES) and write-once-read-many (WORM) storage are used to protect the confidentiality and immutability of data at rest. Blockchain technology is used to achieve decentralized and tamper-proof data storage.

[0004] However, the aforementioned existing technologies have the following technical problems when applied to high-throughput streaming data scenarios in scientific research instruments: Existing technologies pose multiple risks of plaintext exposure, making it difficult to guarantee the confidentiality of data throughout its entire lifecycle. Current solutions typically employ temporary plaintext caching at various stages, including data acquisition cards, transmission protocol stacks, and network reconnection. If the operating system is compromised by malicious programs or a memory dump occurs, this plaintext data is highly susceptible to unauthorized reading, leading to the leakage of critical research data. Existing TLS protocols only protect the transmission pipeline and cannot address the problem of plaintext exposure on endpoint devices.

[0005] Existing technologies protect integrity based on "file" boundaries, which fails to meet the fine-grained verification requirements of streaming data. Traditional file-level signatures or hash verification can only verify whether a complete file has been tampered with. For continuous video streams of microscopic imaging and real-time data streams of gene sequencing, the data is continuously generated and without boundaries. Existing technologies cannot prove the generation order of data blocks, nor can they detect whether intermediate data blocks are lost, rearranged, or subjected to cross-session replay attacks before a complete file is formed. This makes it impossible to confirm the original order and integrity of the data when tracing its origins.

[0006] Traditional blockchain-based evidence storage solutions suffer from performance bottlenecks and excessively high costs, making them unsuitable for handling high-throughput scientific research data. Directly uploading streaming data generated by scientific instruments to the blockchain is insufficient to support real-time data storage at speeds of 200MB / s or even GB / s, due to limitations in the blockchain node's TPS (e.g., approximately 3000 TPS in Hyperledger Fabric) and the high cost of on-chain storage. The existing "full data upload" approach is not feasible from an engineering perspective.

[0007] Existing anchoring strategies lack flexibility and fail to achieve a dynamic balance between real-time performance and cost. To ensure verifiability, current solutions typically anchor data summaries to the blockchain at a fixed frequency (e.g., every 10 seconds). However, network bandwidth in research settings often fluctuates. If the anchoring frequency is too high, it can lead to system congestion and soaring storage costs when bandwidth is limited; if the anchoring frequency is too low, it cannot meet the stringent real-time traceability requirements of research data. Current technologies lack a mechanism that can adaptively adjust the anchoring strategy based on network conditions and cost constraints.

[0008] In network outage scenarios, existing technologies lack secure and consistent retransmission and recovery mechanisms. Research environments (such as field stations and marine research platforms) are unstable, with frequent intermittent outages. Current solutions typically cache data locally during outages, but this cached data is often in plaintext, posing a long-term risk of leakage. After network recovery, existing technologies struggle to quickly distinguish between the original data to be retransmitted and data that may have been tampered with or replayed, potentially leading to the introduction of illegal data during retransmission and compromising the integrity and reliability of the overall dataset.

[0009] In summary, how to achieve an integrated solution for zero-plaintext delivery, block-level verifiability, low-cost anchoring, adaptive adjustment, and secure retransmission during network outages in high-throughput streaming data scenarios of scientific instruments is a technical challenge that urgently needs to be solved in this field. Summary of the Invention

[0010] The technical problem to be solved by this invention is how to achieve zero plaintext storage, block-level verifiability, low-cost anchoring, adaptive adjustment, and secure retransmission after network outage. In order to overcome the defects of the above-mentioned existing technologies (or related technologies), this invention provides a streaming data block-level evidence chain encryption and storage system and method for scientific research instruments.

[0011] This invention provides a streaming data block-level evidence chain encryption and storage system for scientific research instruments, comprising: The scientific research data output module is used to output continuous byte stream data; An encryption device, connected to the scientific data output module, is used to receive the byte stream data and perform the following operations within a trusted execution environment: The byte stream data is stream-divided into multiple data blocks according to the content definition and segmentation rules. Calculate the hash value of each data block and construct a block-level evidence chain based on the hash value, the chain value of the preceding data block, the block sequence number, the session identifier, and the time-series challenge issued by the server. Each of the data blocks is authenticated and encrypted to generate a ciphertext block; Generate a list containing session identifiers, block number ranges, and evidence chain tail values, and sign the list; Generate a summary of the block fingerprint set used to characterize the fingerprints of the received data blocks; The data management module, connected to the encryption device, is used to perform the following operations: Based on the received ciphertext block, the list, the list signature, and the block fingerprint set digest, the session is subjected to replay attack and / or data gap detection. After the detection is successful, a Merkle tree is constructed from the hash value of the received data block. The anchoring period is dynamically selected based on preset traceability delay constraints and cost constraints; The root hash of the Merkle tree and the list signature are written as evidence records into immutable storage; Generate and return a confirmation code to the encryption device; The encryption device is further configured to perform a secure erase / write operation on the confirmed ciphertext block after receiving and verifying the confirmation code.

[0012] Compared with existing technologies, the streaming data block-level evidence chain encryption and storage system for scientific instruments of this invention has the following advantages: This invention utilizes an encryption device to complete all operations from block segmentation and encryption to signing within a trusted execution environment, ensuring that the original byte stream data is encrypted immediately upon generation. This fundamentally avoids the risks of plaintext caching and leakage inherent in traditional solutions at the acquisition card, transmission protocol stack, or offline caching stages. By constructing a block-level evidence chain containing multi-dimensional constraints such as block sequence number, session identifier, and timing challenge, the granularity of data integrity protection is refined from the "file level" to the "block level." This effectively detects data block loss, rearrangement, and cross-session replay, solving the problem that traditional file-level signatures cannot perform sequential verification of streaming data. The invention employs "edge-side streaming block processing + blockchain-only..." The Merkle root hash's separate architecture avoids directly uploading massive amounts of raw data to the blockchain, solving the performance bottlenecks and high costs of traditional blockchain evidence storage solutions in high-throughput scientific research data scenarios. The data management module dynamically selects the anchoring period based on traceability latency and cost constraints, resolving the technical contradiction that fixed anchoring frequencies are either too costly or unable to meet real-time verification requirements under bandwidth fluctuations. Finally, the confirmation code drives secure erasure, ensuring that data is only cleared from the local cache after it has been successfully anchored to immutable storage, solving the security issues of data loss, replay data contamination, and long-term plaintext cache retention in offline scenarios.

[0013] In one possible implementation, the data management module is further configured to maintain a sequence window for the session, the sequence window being used to record the highest consecutive received block sequence number and a bitmap of received data blocks. When the data block is received, the data management module performs the following judgment based on the sequence window: If the sequence number of the newly received data block is not greater than the sequence number of the highest consecutive received block and the bitmap corresponding to the data block is 1, then it is determined to be a replay attack and writing is refused. If the sequence number of the newly received data block minus 1 is greater than the sequence number of the highest consecutive received block, then a data gap is determined and a retransmission is triggered.

[0014] Compared with existing technologies, the above technical solution can identify any attempt to repeatedly submit received data blocks in real time and accurately by maintaining the highest consecutive received block sequence number and the received bitmap, as well as perform compliance judgment on out-of-order data blocks, significantly enhancing the real-time attack detection capability of the session. Furthermore, by utilizing the sequence number continuity rule that the sequence number of a data block minus 1 is greater than the highest consecutive received block sequence number, data gaps in the transmission process can be immediately detected, and the sequence number range of missing blocks can be accurately located. This provides precise triggering conditions and target lists for subsequent retransmission processes, avoiding blind retransmission.

[0015] In one possible implementation, the encryption device employs a producer-consumer concurrent architecture, which includes a zero-copy circular buffer for directly writing the byte stream data, at least one processing thread for performing block-level evidence chain construction, at least one encryption thread for performing authentication encryption and signing, and a batch processing thread for performing data upload.

[0016] Compared with existing technologies, the above technical solution can decouple and process data acquisition, segmentation, encryption, and uploading in parallel through a zero-copy circular buffer and a producer-consumer concurrent architecture. This effectively avoids single-point blocking and ensures that the system can stably handle the continuous data output of scientific instruments at speeds of up to hundreds of MB / s, meeting real-time requirements. Furthermore, the zero-copy technology reduces the number of times data is copied between the kernel and user space, reducing CPU and memory bandwidth consumption. Parallel processing maximizes the utilization of multi-core processor capabilities and shortens the total latency from data generation to encryption completion.

[0017] This invention provides a method for encrypted evidence storage of streaming data at the block level for scientific research instruments, applied to the aforementioned encrypted evidence storage system for streaming data at the block level, comprising the following steps: Step S1: Establish a session with the data management module to obtain the session identifier and the timing challenge issued by the server; Step S2: Obtain the input byte stream data and divide it into blocks using content definition block rules. The content definition block rules include calculating the fingerprint value of the data within the window by sliding a preset sliding window on the byte stream data. When the fingerprint value meets the preset boundary judgment rules, the data block is obtained at the current position. Step S3: Calculate the hash value of each data block and construct a block-level evidence chain based on the hash value, the chain value of the preceding data block, the block sequence number, the session identifier, and the time-series challenge issued by the server. Step S4: When the preset conditions are met, generate a list containing the session identifier, block sequence number range, and evidence chain tail value, and use the device private key to sign the hash value of the list to obtain the list signature; Step S5: Generate a block fingerprint set digest, which is used to record the identity information of the processed data blocks in the current session.

[0018] Compared with existing technologies, the present invention provides a method for encrypted evidence chain storage of streaming data at the block level for scientific research instruments, which has the following advantages: This invention organically integrates key innovative steps such as session establishment, streaming segmentation, multi-dimensional block-level evidence chain construction, list signing, and block fingerprint set digest generation to form a complete technical closed loop from data access to evidence preparation. The resulting block-level evidence chain, list signature, and block fingerprint set digest provide the necessary data foundation and verification basis for subsequent operations such as replay detection, gap retransmission, and adaptive anchoring performed by the data management module, achieving zero plaintext landing, block-level verifiability, low-cost anchoring, adaptive adjustment, and secure retransmission when the network is disconnected.

[0019] In one possible implementation, the sliding window in step S2 is 48 to 128 bytes in size to match the content boundary features of the byte stream data, and the size of the data block is limited between a minimum block length of 64KB to 256KB and a maximum block length of 2MB to 4MB.

[0020] Compared with existing technologies, the above-mentioned technical solution can limit the sliding window size to 48-128 bytes, which is specifically optimized for the inherent content boundary characteristics of scientific data such as pixel blocks in microscopic imaging, reads in gene sequencing, and events in high-energy physics. This makes the block results physically meaningful and improves the verification granularity and credibility of the block-level evidence chain. By setting the minimum and maximum block lengths, it avoids both excessively fine block granularity leading to an excessively long evidence chain and a surge in storage overhead, and excessively large block granularity reducing the accuracy of gap detection. This achieves an optimized balance between fine-grained verification and system performance.

[0021] In one possible implementation, after performing step S5, a network disconnection retransmission process is further included, which includes the following steps: Step A1: Maintain a first fingerprint set for the established session. The first fingerprint set is used to record the identity information of the data block that the server has successfully received. Step A2: After the network is restored, receive an externally input session recovery request. The session recovery request includes a second fingerprint set, which is used to record the identity information of the data block generated on the device side. Step A3: Calculate the difference metric between the first fingerprint set and the second fingerprint set; Step A4: Compare the difference metric value with a preset threshold. If the difference metric value is not greater than the preset threshold, determine the list of data blocks that need to be retransmitted based on the difference metric value, and send a retransmission request containing the list of data blocks to the encryption device. If the difference metric value is greater than the preset threshold, determine that the current session recovery request is abnormal.

[0022] Compared with existing technologies, the above technical solution can provide an efficient set theory-based negotiation mechanism for data retransmission after network outage by introducing a difference metric between the first and second fingerprint sets, ensuring that only missing data blocks are transmitted and maintaining the integrity and consistency of the dataset.

[0023] In one possible implementation, the block fingerprint set summary is a Bloom filter or a bitmap. When the block fingerprint set summary is a bitmap, the difference metric is obtained in step A3 using the following formula:

[0024] in, This represents the difference measurement value; This indicates the number of 1s in the difference bitmap; This represents the second set of fingerprints; This represents the XOR operation; This represents the first set of fingerprints; Indicates the length of the bitmap.

[0025] In one possible implementation, after performing step S5, an adaptive anchoring process is further included, which includes the following steps: Step B1: Obtain the current status information of the data management module, the status information including at least the available network bandwidth and the data block arrival rate; Step B2: Based on the preset traceability delay constraints and cost constraints, and combined with the state information, dynamically select an anchoring period from a predefined set of anchoring period candidates. Step B3: During the anchoring period, construct a Merkle tree from the hash values ​​of the received data blocks and generate a root hash; Step B4: Write the evidence storage record into immutable storage, wherein the evidence storage record includes at least the root hash and the list signature; Step B5: Generate and return a confirmation code to the encryption device, so that the encryption device can perform a secure erase / write operation on the confirmed ciphertext block after receiving and verifying the confirmation code.

[0026] Compared with existing technologies, the above technical solution can concretize the abstract adaptive anchoring algorithm into an executable process. By collecting system status such as available network bandwidth and data block arrival rate in real time, and dynamically selecting the anchoring period by combining traceability delay constraints and cost constraints, it can automatically adapt to the unstable network environment of the scientific research site and always maintain the best operating state. At the same time, the generation and return mechanism of the confirmation code is clarified, and the anchoring operation of the data management module is tightly coupled with the secure erasure operation of the encryption device. This ensures that the device side will be driven to clear the cache only after the data is successfully and irreversibly anchored, forming a complete technical closed loop and eliminating the risk of data loss or premature erasure.

[0027] In one possible implementation, in step B2, candidate periods whose anchoring time is not greater than the traceback delay constraint or whose anchoring cost is greater than the cost constraint are removed from the anchoring period candidate set, and one of the remaining candidate periods is selected as the anchoring period.

[0028] Compared with existing technologies, the above technical solution can forcefully eliminate candidate cycles that do not meet the traceability delay and cost constraints, ensuring that no matter how the state information changes, the selected anchoring strategy must first meet the bottom line requirements of scientific research data for traceability timeliness and economic cost, thus avoiding sacrificing core indicators in pursuit of performance. Attached Figure Description

[0029] Figure 1 This is a schematic diagram of the system structure of the present invention; Figure 2 This is a flowchart illustrating the steps of the method of the present invention; Figure 3 This is a flowchart of the steps in the offline data transfer process of the present invention; Figure 4 This is a flowchart of the adaptive anchoring process of the present invention. Detailed Implementation

[0030] First, those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art can make adjustments as needed to adapt to specific application scenarios.

[0031] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0032] See Figure 1This invention discloses a streaming data block-level evidence chain encryption and storage system for scientific research instruments, comprising a scientific data output module, an encryption device, and a data management module. The scientific data output module specifically includes the scientific instrument and its associated data acquisition card, such as a transmission electron microscope (TEM), a gene sequencer (Illumina NovaSeq), or a high-energy physics detector (CERN LHC type). This module continuously outputs the acquired raw byte stream data to the encryption device in real time via a high-speed physical interface (such as PCIe 3.0 / 4.0, 10GbE fiber). To ensure timing determinism, the interface driver layer is configured in direct memory access (DMA) mode to reduce CPU intervention. The encryption device is deployed beside the scientific instrument and is a device equipped with a Trusted Execution Environment (TEE, such as Intel SGX or ARM). The embedded edge computing unit or dedicated security gateway of TrustZone contains a secure key storage unit, a streaming engine, a ciphertext cache unit, and a network communication unit. As the security boundary of the entire system, the encryption device ensures that the original byte stream data exists only briefly in plaintext form within the trusted execution environment after leaving the scientific instrument, and remains in ciphertext state at all other times. The data management module is deployed in a data center or cloud and can consist of a high-performance server cluster and a distributed storage system. This data management module is responsible for data verification, aggregation, anchoring, and long-term storage.

[0033] In this embodiment of the invention, the encryption device processes the input byte stream data through the following internal process: Data Ingestion and Zero-Copy Buffer: The encryption device maintains a zero-copy circular buffer that is directly written to by DMA. The byte stream data output by the data acquisition card is directly written to this circular buffer through kernel bypass technology (such as DPDK), avoiding multiple copies of data between kernel space and user space, reducing CPU load and processing latency. The size and number of slots in the circular buffer are configured according to the preset maximum data rate (such as 400MB / s) and average processing latency to prevent the circular buffer from overflowing. Streaming Blocking and Evidence Chain Construction: One or more block processing threads continuously consume byte stream data from the circular buffer, perform Content Definition Blocking (CDC) and block-level evidence chain construction. The core logic of the block processing threads is detailed in the method implementation examples below. Authentication and Encryption: After data blocks are segmented, they are passed to the encryption thread pool. The encryption thread pool uses the session key to perform AES-GCM authentication and encryption on each data block, outputting ciphertext blocks. and certification labels The AES-GCM algorithm can provide both data confidentiality and integrity protection. Its Additional Authentication Data (AAD) can include block sequence number i and session identifier sid to bind the ciphertext block to its metadata. Inventory Generation and Fingerprint Digest: At the end of a session or a preset sliding window (such as one linked to the anchoring period), the main control thread generates an inventory M and signs it with the device private key sk_dev stored inside the TEE using ECDSA, obtaining the inventory signature Sig_M. Simultaneously, the hash value of the processed data blocks is used... Generate a block fingerprint set digest (BF). The block fingerprint set digest can be a Bloom filter or a fixed-length bitmap. When a bitmap is used, the block number i is directly mapped to the (i mod m)th bit of the bitmap. Data Upload and Confirmation: The upload batch processing thread encapsulates data such as ciphertext blocks, block hash values, lists and list signatures, and fingerprint digest sets into a batch according to the strategy of "maximum batch size (e.g., reaching 100 blocks or 16MB) or maximum waiting time (e.g., 500ms)" and uploads it to the data management module through a TLS encrypted channel. After uploading, the batch of data is temporarily stored in the local cache, waiting for the confirmation code.

[0034] In this embodiment of the invention, the data management module processes data from the encryption device through the following process: Session-level admission control and replay detection: After receiving data, the access gateway first looks up the corresponding session context based on the session identifier sid. The context maintains a sequence window W_seq, which records the highest consecutive received block sequence number i_max of the current session and a fixed-length receive bitmap RecvMap (e.g., a 1024-bit circular bitmap that records the block reception status within the range from i_max-W_seq+1 to i_max). Based on this sequence window, replay attacks and data gaps are identified. Block fingerprint set maintenance: The server maintains a first block fingerprint set BF_srv for each active session. When a new data block passes the replay test, its hash value is set. (Or block number i) is inserted into the first fingerprint set. The data structure of the first fingerprint set is consistent with that of the second fingerprint set on the device side (both are Bloom filters or bitmaps). Merkle Tree Construction and Adaptive Anchoring: Using a statistical time slice T_slot (e.g., 1 second) as the period, the current available network bandwidth BW is collected, the data block arrival rate λ is calculated, and historical anchoring costs are retrieved. Based on this real-time status information and preset business constraints (tracing back latency constraint T_max and cost constraint C_max), the optimal anchoring period Δ is dynamically selected from a predefined candidate set (e.g., {1s, 5s, 10s, 30s, 60s, 300s}). After selection, all received hash values ​​are aggregated within the anchoring period Δ to construct a Merkle tree, and the root hash R_Δ* is calculated. Evidence storage and confirmation: {R_Δ*, Sig_M, sid, anchor timestamp} is written as an evidence storage record to immutable storage. Immutable storage can be object storage that supports WORM features (such as AWS S3 Object Lock) or evidence storage service of consortium blockchain. After successful writing, the confirmation code ack = H(R_Δ* || sid || nonce_srv) is calculated and returned to the corresponding encryption device through a secure connection. Secure erase / write driver: After receiving the acknowledgment code ack, the encryption device recalculates the acknowledgment code ack using the locally stored root hash R_Δ*, session identifier sid, and server-signed timing challenge nonce_srv, and compares them. If they match, it proves that the batch of data has been successfully anchored. The encryption device then updates the local commit bitmap AckMap, marking all data blocks in the batch as "acknowledged". Afterward, according to the strategy (such as immediate erase or timed batch erase), a secure erase / write operation is performed on the acknowledged ciphertext blocks. Specifically, this includes: repeatedly overwriting the ciphertext buffer area with random numbers or destroying the session key used to encrypt the batch of data, so that the remaining ciphertext cannot be decrypted.

[0035] See Figure 2 The present invention also discloses a method for encrypted evidence chain storage of streaming data at the block level for scientific instruments, comprising the following steps: Step S1: Establish a session with the data management module to obtain the session identifier and the timing challenge issued by the server; Step S2: Obtain the input byte stream data and divide it into blocks using content-defined block division rules. The content-defined block division rules include using a preset sliding window to slide across the byte stream data to calculate the fingerprint value of the data within the window. When the fingerprint value meets the preset boundary judgment rules, the data block is obtained at the current position. Step S3: Calculate the hash value of each data block and construct a block-level evidence chain based on the hash value, the chain value of the preceding data block, the block sequence number, the session identifier, and the time-series challenge issued by the server. Step S4: When the preset conditions are met, generate a list containing session identifier, block sequence number range, and evidence chain tail value, and use the device private key to sign the hash value of the list to obtain the list signature. Step S5: Generate a block fingerprint set digest. The block fingerprint set digest is used to record the identity information of the data blocks that have been processed in the current session.

[0036] In this embodiment of the invention, in step S1, after the encryption device powers on and establishes an initial connection with the data management module, it performs two-way authentication. The encryption device sends its device certificate, and the data management module verifies the validity of the certificate. After successful verification, the data management module assigns a globally unique session identifier sid to this session and generates a high-entropy random number as a timing challenge nonce_srv, which is also sent to the encryption device. At the same time, both parties negotiate the encryption algorithm suite (such as AES-256-GCM, SHA-256) and anchoring policy constraint parameters (T_max, C_max) to be used in this session. The data management module generates a session key and encapsulates it with the public key of the encryption device (such as RSA-OAEP) to obtain the encapsulated session key E_K, which is then sent out. The encryption device uses its private key to decapsulate the session key within the TEE to obtain the session key. At this point, the session context (sid, nonce_srv, K, T_max, C_max, etc.) is established within the TEE of the encryption device.

[0037] In this embodiment of the invention, in step S2, the block-splitting thread of the encryption device reads byte stream data from the zero-copy circular buffer and performs CDC block splitting. The specific algorithm is as follows: Initialize the current data block start pointer p_start and the current data block size current_len = 0; Define the sliding window size W = 48~128 bytes (the specific value can be configured according to the instrument type: 48-64 for TEM / SEM, 64-96 for gene sequencing, and 96-128 for high energy physics). Define a mask, which determines the average size of the blocks. The expected average block size is on the order of 2^popcount(mask). Starting from p_start, a sliding window with a step size of 1 byte is used to calculate the Rabin fingerprint fp=rolling_hash(window) on the data within the window; Boundary determination rule: if (fp & mask) == 0, then the current position is determined to be a block boundary; Enforce block size limit: If current_len reaches the minimum block length L_min (e.g., 64KB) and the boundary conditions are met, then the block is cut. If current_len reaches the maximum block length L_max (e.g., 2MB) but has not yet encountered a boundary, then a block is forcibly cut at the current position; After determining the boundary position p_end, the data block B_i = S[p_start : p_end] is segmented, the block number i is incremented by 1, p_start = p_end + 1 is updated, current_len is reset, and the scanning continues.

[0038] In this embodiment of the invention, in step S3, for each segmented data block B_i, the session identifier sid and the timing challenge nonce_srv in the session context, as well as the chain value c_{i-1} of the previous data block (c_0 is a salt value generated by a secure random number), are read. Then, a block-level evidence chain c_i = SHA-256(c_{i-1} || h_i || i || sid || nonce_srv) is constructed, and (i, h_i, c_i) and B_i (or the encrypted c_i) are stored in the queue to be processed for consumption by subsequent encryption threads.

[0039] In this embodiment of the invention, during step S4, the main control thread of the encryption device monitors the increase of the block sequence number, and triggers list generation when any of the following preset conditions are met: The cumulative number of blocks reaches a preset value (e.g., 10,000 blocks), the cumulative data volume reaches a preset value (e.g., 1GB), and the time elapsed since the last list was generated reaches a preset value (e.g., linked to the current anchoring period Δ, or a fixed value such as 10 seconds). Generate a list whose data structure is a structured dictionary, containing, but not limited to, the following fields: sid: Current session identifier; dev_id: Unique device identifier; i_start, i_end: Sequence numbers of the first and last data blocks covered by this list; c_tail: Evidence chain value of the last data block (i.e., the data block corresponding to i_end); salt: c_0 value used to initialize the evidence chain; alg_id: Hash algorithm (e.g., "sha256") and signature algorithm (e.g., "ecdsa-p256") used; pk_id: Public key ID or certificate fingerprint corresponding to the device private key used for signing; t_start, t_end: Start and end timestamps of the data blocks covered by this list; nonce_srv: Time challenge issued when the session is established, used to bind the list to the session. The hash value H(M) of the list is calculated as SHA-256(M). Inside the TEE, the device private key sk_dev is used to perform an ECDSA signature on H(M) to obtain the signature value Sig_M.

[0040] In this embodiment of the invention, in step S5, the block thread or another lightweight thread is responsible for maintaining the block fingerprint set digest BF. Taking a bitmap implementation as an example, a second block fingerprint set BF_dev with all zeros and a length of m bits (e.g., m=1,000,000) is initialized. For each newly generated block number i, its position p = i mod m in the second block fingerprint set is calculated, and the p-th position of the second block fingerprint set BF_dev is set to 1. When the list M is generated, the snapshot of the current second block fingerprint set BF_dev is used as the block fingerprint set digest corresponding to the list.

[0041] See Figure 3 In this embodiment of the invention, after step S5, a network disconnection retransmission process is further included, which includes the following steps: Step A1: Maintain the first fingerprint set for the established session. The first fingerprint set is used to record the identity information of the data blocks that the server has successfully received. Step A2: After the network is restored, receive an externally input session recovery request. The session recovery request contains a second fingerprint set, which is used to record the identity information of the data blocks generated on the device side. Step A3: Calculate the difference metric between the first fingerprint set and the second fingerprint set; Step A4: Compare the difference metric value with a preset threshold. If the difference metric value is not greater than the preset threshold, determine the list of data blocks that need to be retransmitted based on the difference metric value, and send a retransmission request containing the list of data blocks to the encryption device. If the difference metric value is greater than the preset threshold, determine that the current session recovery request is abnormal.

[0042] In this embodiment of the invention, during the session persistence process in step A1, the data management module maintains two core data structures for each session identifier sid: a first fingerprint set BF_srv (corresponding to the data blocks that have been persistently stored on the server) and a sequence window W_seq (corresponding to the data blocks that have been received but may not yet be anchored). The first fingerprint set BF_srv is updated when the data of a data block is successfully written to the immutable underlying file system, and its hash value or block sequence number is inserted into the first fingerprint set BF_srv.

[0043] In this embodiment of the invention, during step A2, assuming the network at the research site is interrupted, the encryption device continues to generate and process data, persistently storing all ciphertext blocks, lists, block fingerprint set digests, etc., in the local encrypted storage area (non-plaintext). After the network is restored, the encryption device actively initiates a session recovery request to the data management module, i.e., sends a HELLO message. The HELLO message includes at least: sid: original session identifier, i_max_dev: sequence number of the last data block generated and cached on the device side, BF_dev: summary of the block fingerprint set currently maintained on the device side, nonce_srv: time-series challenge of the original session, used to prove that the request belongs to the original session, H(M_last) or Sig_M_last: hash value or signature of the last list, used to quickly verify the session state.

[0044] In this embodiment of the invention, in step A3, after the server receives the HELLO message, it first verifies the consistency between the timing challenge nonce_srv and the local record, and whether H(M_last) matches the local record. If the verification fails, the session is directly determined to be abnormal. If the verification passes, the server retrieves the first fingerprint set BF_srv maintained locally. Assuming that the second fingerprint set BF_dev and the first fingerprint set BF_srv are both implemented using bitmaps of length m, the server calculates the difference metric value using the following formula:

[0045] in, This represents the XOR operation. This indicates the number of 1s in the difference bitmap. This represents the second set of fingerprints. This represents the first set of fingerprints; Then, in step A4, the difference measure Δ_BF is compared with a preset threshold θ_BF (e.g., 0.05, or 5%).

[0046] In this embodiment of the invention, step A4 includes hierarchical processing, which includes normal retransmission and abnormal alarm and full verification. In the normal retransmission scenario, the server considers the difference metric value to be within an acceptable range. Based on the difference metric values ​​of BF_dev and BF_srv, and combined with the locally recorded block metadata, the server generates a precise list of block indices [i_list] that needs to be retransmitted. The server returns a NEED message containing sid, [i_list], and i_max_srv (the maximum sequence number continuously received by the server). Upon receiving this message, the encryption device only retrieves the data block corresponding to [i_list] from its local cache for processing. In scenarios involving abnormal alerts and full verification, if the server determines that the current session recovery request is highly suspicious, possibly due to local cache tampering, severe data loss, or malicious attacks, the server will not allow incremental retransmission. Instead, it will mark the session status as "suspicious" and refuse any new writes. The server will issue an alert and initiate a full verification process. Full verification forces the encryption device to upload all data blocks and their lists from a certain starting block to i_max_dev. The server will perform hash verification, evidence chain recalculation, and signature verification on these data one by one. Before the verification is passed, all data in the session is considered untrusted.

[0047] See Figure 4 In this embodiment of the invention, after step S5, an adaptive anchoring process is further included, which includes the following steps: Step B1: Obtain the current status information of the data management module. The status information includes at least the available network bandwidth and the data block arrival rate. Step B2: Based on the preset traceability delay constraints and cost constraints, and combined with the status information, dynamically select an anchoring period from a predefined set of anchoring period candidates. Step B3: Construct a Merkle tree and generate a root hash from the hash values ​​of the received data blocks within the anchoring period; Step B4: Write the evidence record to immutable storage. The evidence record must include at least the root hash and the list signature. Step B5 generates and returns a confirmation code to the encryption device, which then drives the encryption device to perform a secure erase / write operation on the confirmed ciphertext block after receiving and verifying the confirmation code.

[0048] In this embodiment of the invention, in step B1, the following collection operation is performed once in each scheduling cycle (e.g., 1 second): obtain the current available network bandwidth BW (unit: bps), which reflects the uplink congestion level; obtain the data block arrival rate λ (unit: blocks / second) in the past T_slot; obtain the cost per anchor number cost_per_anchor, and calculate the actual anchor cost over the past period, and estimate the cost C_est(Δ) that may be incurred if different anchor periods are used.

[0049] In this embodiment of the invention, in step B2, a predefined set of anchoring period candidates is maintained, for example, Δ_candidates = [1, 5, 10, 30, 60, 300], in seconds. For each candidate period Δ, the following filtering logic is executed: Real-time constraint: Δ ≤ T_max. If Δ is greater than T_max (for example, the business requires that the data be traceable for a maximum of 1 minute after it is generated, while Δ = 300 seconds obviously does not meet the requirement), then the candidate will be removed. Cost constraint: Estimate the anchoring cost C_est(Δ). C_est(Δ) is calculated as (time_unit / Δ) * cost_per_anchor. If C_est(Δ) > C_max (for example, the budget limit is 1000 yuan per month, while the estimated cost of Δ=1 second is far beyond this value), then the candidate period is eliminated. After the above two-layer screening, a candidate periodicity list Δ_feasible that satisfies all hard constraints is obtained. If Δ_feasible is empty, the process proceeds to degeneracy. If Δ_feasible is not empty, then the optimization rule is applied: select the smallest candidate period from Δ_feasible that minimizes system overhead and satisfies the real-time traceability requirement. System overhead is positively correlated with anchoring frequency, so "smallest candidate period" means selecting the period with the smallest value (i.e. the highest frequency) under the premise of meeting cost constraints. This can maximize the real-time traceability of data within the budget. For example, if {5, 10, 30} all meet the constraints, then the anchoring period Δ = 5s is selected.

[0050] In this embodiment of the invention, in step B3, after selecting the anchoring period, all received block hash values ​​that have passed the replay detection are used as leaf nodes within a time window to construct a binary Merkle tree and calculate the root hash. During the construction process, all intermediate hash values ​​are persistently stored for subsequent traceability verification.

[0051] In this embodiment of the invention, to enable those skilled in the art to more clearly understand the invention, a complete end-to-end workflow example is provided. Assuming the scenario involves gene sequencer data storage and a brief network interruption, the example workflow is as follows: Session establishment: The sequencer starts up, the encryption device establishes a secure connection with the data management module in the cloud, and obtains the session identifier sid="seq_run_001", the time sequence challenge nonce_srv issued by the server="0x7a2f...", and the session key K; Data processing and uploading: The sequencer outputs a FASTQ data stream. The encryption device divides the CDC into blocks using a sliding window of W=64 bytes, with block sizes ranging from 150KB to 1.5MB. It calculates the hash value h_i and the ciphertext block c_i, and uses the session key for AES-GCM encryption. Every 1000 blocks processed, a list M is generated and signed, and the fingerprint set BF_dev of the second block is updated. The upload thread uploads data every 500ms or after accumulating 50 blocks. Network outage occurs: The network interruption lasts for 3 minutes. The encryption device continues to process data. All ciphertext blocks, lists, list signatures Sig_M, and the second fingerprint set BF_dev are securely stored in the local encryption cache. The encryption device detects the upload failure and marks the data to be uploaded as "to be retransmitted". Network disconnection recovery: Once the network connection is restored, the encryption device immediately sends HELLO(sid="seq_run_001", i_max_dev=15000, BF_dev, nonce_srv) to the server. Consistency check: The server finds the session with sid="seq_run_001", retrieves the local BF_srv (already recorded in block number 14800), calculates the difference metric Δ_BF=0.01, which is less than the preset threshold θ_BF=0.05. Based on the difference metric, the server calculates the missing block list [14801, 14802, ..., 15000], and returns NEED(sid, list=[14801...15000]). Precise retransmission: The encryption device only reads blocks 14801~15000 and their corresponding metadata from the cache and re-uploads them; Adaptive anchoring: After the server monitors that the network has recovered and the bandwidth is sufficient, the data block arrival rate λ increases. Based on the traceability latency constraint T_max=60s and the cost constraint C_max=￥0.01 / time, Δ*=10s is dynamically selected (instead of the 30s before the network outage) to improve the real-time performance of traceability. Merkle Anchoring and Acknowledgment: Over the next 10 seconds, the server aggregates the received block hashes, constructs a Merkle tree to obtain the root hash R, writes {R, Sig_M, ...} into the WORM storage, generates an acknowledgment code ack, and returns it to the encryption device. Secure erase / write: After the encryption device verifies that the ACK is correct, it securely erases the corresponding blocks 14801~15000 from the local encryption cache (overwriting with a random number). Retrospective verification: One year later, the researchers needed to verify the authenticity of the sequencing data. They obtained the anchored record and list signature from the immutable storage, downloaded the encrypted data from the archive storage, and confirmed that the data had not been tampered with or rearranged in any way since its generation by verifying the signature, recalculating the chain of evidence, and verifying the Merkle proof. After authorization, they used the session key to decrypt and obtain the original FASTQ data stream.

[0052] In the description of this invention, the references to "one embodiment," "some embodiments," "in this embodiment," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0053] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A streaming data block-level evidence chain encryption and storage system for scientific research instruments, characterized in that, include: The scientific research data output module is used to output continuous byte stream data; An encryption device, connected to the scientific data output module, is used to receive the byte stream data and perform the following operations within a trusted execution environment: The byte stream data is stream-divided into multiple data blocks according to the content definition and segmentation rules. Calculate the hash value of each data block and construct a block-level evidence chain based on the hash value, the chain value of the preceding data block, the block sequence number, the session identifier, and the time-series challenge issued by the server. Each of the data blocks is authenticated and encrypted to generate a ciphertext block; Generate a list containing session identifiers, block number ranges, and evidence chain tail values, and sign the list; Generate a summary of the block fingerprint set used to characterize the fingerprints of the received data blocks; The data management module, connected to the encryption device, is used to perform the following operations: Based on the received ciphertext block, the list, the list signature, and the block fingerprint set digest, the session is subjected to replay attack and / or data gap detection. After the detection is successful, a Merkle tree is constructed from the hash value of the received data block. The anchoring period is dynamically selected based on preset traceability delay constraints and cost constraints; The root hash of the Merkle tree and the list signature are written as evidence records into immutable storage; Generate and return a confirmation code to the encryption device; The encryption device is further configured to perform a secure erase / write operation on the confirmed ciphertext block after receiving and verifying the confirmation code.

2. The streaming data block-level evidence chain encryption and storage system according to claim 1, characterized in that, The data management module is also used to maintain a sequence window for the session. The sequence window is used to record the highest consecutive received block sequence number and a bitmap of received data blocks. When the data management module receives the data block, it performs the following judgment based on the sequence window: If the sequence number of the newly received data block is not greater than the sequence number of the highest consecutive received block and the bitmap corresponding to the data block is 1, then it is determined to be a replay attack and writing is refused. If the sequence number of the newly received data block minus 1 is greater than the sequence number of the highest consecutive received block, then a data gap is determined and a retransmission is triggered.

3. The streaming data block-level evidence chain encryption and storage system according to claim 1, characterized in that, The encryption device employs a producer-consumer concurrent architecture, which includes a zero-copy circular buffer for directly writing the byte stream data, at least one processing thread for performing block-level evidence chain construction, at least one encryption thread for performing authentication encryption and signing, and a batch processing thread for performing data upload.

4. A method for encrypted evidence chain storage of streaming data at the block level for scientific instruments, characterized in that, The method applied to the streaming data block-level evidence chain encryption and storage system as described in any one of claims 1-3 includes the following steps: Step S1: Establish a session with the data management module to obtain the session identifier and the timing challenge issued by the server; Step S2: Obtain the input byte stream data and divide it into blocks using content definition block rules. The content definition block rules include calculating the fingerprint value of the data within the window by sliding a preset sliding window on the byte stream data. When the fingerprint value meets the preset boundary judgment rules, the data block is obtained at the current position. Step S3: Calculate the hash value of each data block and construct a block-level evidence chain based on the hash value, the chain value of the preceding data block, the block sequence number, the session identifier, and the time-series challenge issued by the server. Step S4: When the preset conditions are met, generate a list containing the session identifier, block sequence number range, and evidence chain tail value, and use the device private key to sign the hash value of the list to obtain the list signature; Step S5: Generate a block fingerprint set digest, which is used to record the identity information of the processed data blocks in the current session.

5. The method for encrypted evidence chain storage of streaming data at the block level according to claim 4, characterized in that, In step S2, the sliding window size is 48 to 128 bytes to match the content boundary features of the byte stream data. The size of the data block is limited to a minimum block length and a maximum block length, with the minimum block length being 64KB to 256KB and the maximum block length being 2MB to 4MB.

6. The method for encrypted storage of streaming data block-level evidence chains according to claim 4, characterized in that, After performing step S5, a network disconnection recovery process is also included, which includes the following steps: Step A1: Maintain a first fingerprint set for the established session. The first fingerprint set is used to record the identity information of the data block that the server has successfully received. Step A2: After the network is restored, receive an externally input session recovery request. The session recovery request includes a second fingerprint set, which is used to record the identity information of the data block generated on the device side. Step A3: Calculate the difference metric between the first fingerprint set and the second fingerprint set; Step A4: Compare the difference metric value with a preset threshold. If the difference metric value is not greater than the preset threshold, determine the list of data blocks that need to be retransmitted based on the difference metric value, and send a retransmission request containing the list of data blocks to the encryption device. If the difference metric value is greater than the preset threshold, determine that the current session recovery request is abnormal.

7. The method for encrypted evidence chain storage of streaming data at the block level according to claim 6, characterized in that, The block fingerprint set summary is either a Bloom filter or a bitmap. When the block fingerprint set summary is a bitmap, the difference metric is obtained in step A3 using the following formula: in, This represents the difference measurement value; This indicates the number of 1s in the difference bitmap; This represents the second set of fingerprints; This represents the XOR operation; This represents the first set of fingerprints; Indicates the length of the bitmap.

8. The method for encrypted storage of streaming data block-level evidence chains according to claim 4, characterized in that, After performing step S5, an adaptive anchoring process is further included, which includes the following steps: Step B1: Obtain the current status information of the data management module, the status information including at least the available network bandwidth and the data block arrival rate; Step B2: Based on the preset traceability delay constraints and cost constraints, and combined with the state information, dynamically select an anchoring period from a predefined set of anchoring period candidates. Step B3: During the anchoring period, construct a Merkle tree from the hash values ​​of the received data blocks and generate a root hash; Step B4: Write the evidence storage record into immutable storage, wherein the evidence storage record includes at least the root hash and the list signature; Step B5: Generate and return a confirmation code to the encryption device, so that the encryption device can perform a secure erase / write operation on the confirmed ciphertext block after receiving and verifying the confirmation code.

9. The method for encrypted evidence chain storage of streaming data at the block level according to claim 8, characterized in that, In step B2, candidate periods whose anchoring time is not greater than the traceback delay constraint or whose anchoring cost is greater than the cost constraint are removed from the candidate anchoring period set, and one of the remaining candidate periods is selected as the anchoring period.

Images