Archive digital full-process management system and method based on block chain

Abstract

The invention discloses an archive digitization full-process management system and method based on a block chain. The system comprises an archive input module, a digitization processing module, a hash calculation module, a metadata management module, a block chain interface module, a storage module, a verification module, an audit tracking module and a user authority management module. According to the system, a physical file is converted into a digital file, after preprocessing and OCR recognition, an SHA-256 hash value is calculated, hash and metadata are uploaded to a block chain network, and meanwhile the digital file is stored in an IPFS in a distributed mode; according to the method, the Hash value and the key metadata of the archive digital file are stored in the non-tampering block chain, the original file is stored by using the distributed storage system IPFS, and any tampering on the stored digital file can cause that the Hash value of the stored digital file is not consistent with the original value recorded on the block chain, so that the stored digital file is detected; the problem that data is easily tampered or lost in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and in particular to a blockchain-based full-process management system and method for digitizing archives. Background Technology

[0002] Archival digitization process management is the process of converting physical archives into digital form through technical means, and achieving efficient and secure management and utilization. Existing archival digitization systems typically adopt a traditional centralized architecture, involving components such as scanning equipment, image processing software, database storage, and access control modules to realize the digitization, storage, and retrieval of archives. These systems usually rely on centralized servers for data management and verification, and use conventional encryption and backup mechanisms to ensure data security.

[0003] First, because the system relies on centralized storage and verification mechanisms, digitized archival data is easily tampered with or lost, and effective integrity verification is difficult to achieve. Second, operation logs and metadata in the archival digitization process are usually stored in local databases, lacking transparent and tamper-proof audit trails, making it impossible to provide reliable evidence in case of disputes. Third, access control is mostly based on simple username and password mechanisms, which are susceptible to internal abuse or external attacks, and cannot achieve automated access control. Fourth, traditional systems are inefficient in terms of data sharing and interoperability, and it is difficult to ensure the chronological order and authenticity of the digitization process.

[0004] Therefore, in response to the problems mentioned above, this invention proposes a blockchain-based full-process management system and method for digitizing archives. Summary of the Invention

[0005] To overcome the problems of existing technologies, such as easy data tampering, lack of audit transparency, weak access control, and low sharing efficiency, this invention proposes a blockchain-based full-process management system and method for archival digitization. By leveraging the immutability and distributed nature of blockchain technology, combined with smart contracts, distributed storage, and advanced algorithms, the system achieves automated, secure, and traceable management of the entire archival digitization process.

[0006] The technical solution of this invention is: a blockchain-based end-to-end management system for digitizing archives, comprising: The document input module is used to receive physical documents and convert them into raw digital files through a scanning device, including a high-resolution scanner or digital camera, which supports multiple formats including PDF, JPEG or TIFF. The digitization processing module, connected to the file input module, is used to preprocess the original digital files, including image enhancement, noise removal, and format standardization, and to extract text content through an OCR algorithm. This OCR algorithm is based on a convolutional neural network for character recognition and semantic analysis. It also includes a quality detection unit, which is used to automatically detect the quality indicators of digital files based on image processing algorithms, including resolution, sharpness, and color fidelity. This image processing algorithm uses the Sobel operator for edge detection and PSNR calculation. If the quality does not meet the standards, a re-digitization process is triggered. The hash calculation module, connected to the digitization processing module, is used to calculate the hash value of the preprocessed digital file using the SHA-256 hash algorithm and generate a unique digital fingerprint. The metadata management module is used to generate and store the metadata of archives, including archive number, creation time, archive type, personnel information, and digitization parameters. This metadata is associated with the digital file. The blockchain interface module, connected to the hash calculation module and the metadata management module, is used to upload hash values ​​and metadata to the blockchain network. This blockchain network is based on the Ethereum platform, adopts the proof-of-stake consensus mechanism, and runs smart contracts to execute data uploading and verification logic. A storage module for storing digital files and their metadata includes a distributed storage system based on IPFS, where digital files are stored in IPFS and IPFS hash pointers are stored in the blockchain; The verification module, connected to the blockchain interface module and the storage module, is used to obtain the stored hash value by querying the blockchain network and compare it with the locally recalculated hash value to verify the integrity and authenticity of the digital file. The audit trail module is used to record the operation log of the entire process of document digitization, including scanning time, processing steps and on-chain time, and uploads the log hash value to the blockchain through the blockchain interface module to ensure that the operation is tamper-proof; The user access management module, based on a role-based access control algorithm, manages the access permissions of different users to the system, including data entry clerks, auditors, and administrators, and uses smart contracts to implement the allocation and verification of permissions.

[0007] Preferably, the smart contract in the blockchain interface module includes an archive upload contract and a verification contract. The archive upload contract is used to receive hash values ​​and metadata, package them into transactions, and submit them to the blockchain. The verification contract is used to respond to external queries and return hash value comparison results. The smart contract includes an event triggering mechanism to automatically notify relevant users when data upload or verification fails.

[0008] Preferably, the distributed storage system of the storage module adopts IPFS and combines it with an encryption algorithm to encrypt digital files. The encryption algorithm is based on AES-256 symmetric encryption, and the key is managed through a blockchain smart contract. Only authorized users can obtain the decryption key through the permission management module.

[0009] Preferably, the verification module further includes a timestamp verification unit, used to verify the time sequence of the digitized archives through a timestamp service in the blockchain network. This timestamp service is based on the blockchain block generation time and synchronized with NTP to ensure that the operation time cannot be tampered with.

[0010] This invention provides a blockchain-based method for end-to-end management of digital archives, comprising the following steps: S1 receives physical files through the file input module and converts them into raw digital files using a scanning device; S2 preprocesses the original digital file through the digitization processing module, including image enhancement, noise removal, format standardization, and quality inspection. It also extracts the text content through an OCR algorithm based on CNN for character recognition. The quality inspection includes automatically detecting the quality indicators of the digital file based on the image processing algorithm, including resolution, sharpness, and color fidelity. The image processing algorithm uses the Sobel operator for edge detection and PSNR calculation. If the quality does not meet the standards, a re-digitization process is triggered. S3 uses the hash calculation module to calculate the hash value of the preprocessed digital file using the SHA-256 algorithm, generating a unique digital fingerprint; S4 generates metadata for archives through the metadata management module, including archive number, creation time, and personnel information, and associates the metadata with digital files; S5 uploads hash values ​​and metadata to the blockchain network through the blockchain interface module. This blockchain network is based on the Ethereum platform, adopts the PoS consensus mechanism, and automatically executes data uploading to the blockchain through smart contracts. Data uploading to the blockchain is executed through smart contracts, which include an archive uploading contract and a verification contract. The archive uploading contract is used to receive hash values ​​and metadata, package them into transactions, and submit them to the blockchain. The verification contract is used to respond to external queries and return hash value comparison results. S6 stores digital files in the distributed storage system IPFS through the storage module and stores the IPFS hash pointer in the blockchain. It encrypts the digital files using the AES-256 symmetric encryption algorithm. The key is managed through the blockchain smart contract, and only authorized users can obtain the decryption key through the permission management module. S7. The verification module queries the blockchain network to obtain the stored hash value and compares it with the locally recalculated hash value to verify the integrity and authenticity of the digital file. At the same time, the time sequence of the archive digitization is verified through the timestamp service in the blockchain network. The timestamp service is based on the blockchain block generation time and synchronized with NTP to ensure that the operation time cannot be tampered with. S8 records the entire process operation log through the audit trail module and puts the log hash value on the blockchain to ensure that the operation cannot be tampered with. S9 manages user access permissions based on the RBAC algorithm through the user permission management module, and implements automatic permission verification through smart contracts.

[0011] The beneficial effects of this invention are: 1. This invention stores the hash value and key metadata of digital archival files in an immutable blockchain and uses the distributed storage system IPFS to save the original files. Any tampering with the stored digital files will cause its hash value to be inconsistent with the original value recorded on the blockchain, which will be detected by the verification module. This solves the problem of data being easily tampered with or lost in the prior art and ensures the long-term authenticity and credibility of digital archival content.

[0012] 2. This invention synchronizes the hash values ​​of key operation logs (such as scanning, processing, and uploading to the blockchain) throughout the entire process of digitizing archives to the blockchain through an audit trail module. By utilizing the immutability and time-series characteristics of the blockchain, a transparent and reliable audit trail is constructed. Any participant can independently verify the entire operation history and cannot deny or modify the recorded operations, thereby solving the problems of opaque audit trails and weak evidentiary value in the prior art.

[0013] 3. This invention integrates a permission management module based on role-based access control (RBAC) algorithm with blockchain smart contracts, thereby automating and decentralizing the permission allocation and verification process. The execution of permission policies is controlled by predefined smart contracts, avoiding the risk of abuse that may result from excessive concentration of administrator power in centralized systems.

[0014] 4. By adopting IPFS as the distributed storage foundation and combining it with blockchain-managed encryption keys, this invention significantly improves the efficiency of sharing and utilizing archival data while ensuring data security. Authorized users can quickly locate and access the digital archives they need without going through a central server. At the same time, the automated permission verification and decryption key distribution mechanism of smart contracts simplifies the sharing process. Attached Figure Description

[0015] Figure 1 The diagram shown is a schematic representation of the system framework of the present invention. Figure 2The diagram shown illustrates the workflow of this invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] Please see Figure 1 and Figure 2 The present invention provides an embodiment: (1) The operator places a physical file (such as a historical contract) under the scanning device. The file input module controls the scanning device to generate a high-resolution TIFF format original digital image file. The file is immediately transferred to the digitization processing module. The digitization processing module first performs data preprocessing, uses histogram equalization algorithm to adjust image contrast to ensure clear contrast between text and background; at the same time, it uses median filter to eliminate salt-and-pepper noise generated during scanning, and converts the processed image into PDF format.

[0018] After preprocessing, the OCR (Optical Character Recognition) algorithm is run. This algorithm uses a character recognition model based on a convolutional neural network (CNN). The model is pre-trained using a dataset containing millions of Chinese and English character images and can recognize printed and handwritten fonts.

[0019] The specific process of the above-mentioned OCR (Optical Character Recognition) algorithm includes: firstly, extracting local features (such as stroke edges and corners) in the image through multi-layer convolution and pooling operations of CNN; then, performing sequence modeling through recurrent neural network (RNN) layers combined with connectionist temporal classification (CTC) layers; and finally outputting editable and searchable text content. The extracted text is output together with the PDF file as the final form of digital archive.

[0020] (2) The hash calculation module receives the final version of the PDF file from the digitization processing module. It uses the SHA-256 hash algorithm to process the file as a continuous binary data stream and generate a 64-bit hexadecimal string, which is the unique digital fingerprint of the file (e.g., "4f8e2a1b...c3d9"). This fingerprint is like the "identity card" of the digital file. Any slight change will cause the fingerprint to change drastically. At the same time, the metadata management module automatically generates and encapsulates the metadata of the file to form a structured JSON data packet.

[0021] (3) The blockchain interface module calls the "Archive On-Chain Smart Contract" deployed on the blockchain network (in this embodiment, an Ethereum-compatible chain based on the Proof-of-Stake (PoS) consensus mechanism is used). The module uses the archive ID and file hash value from the metadata as parameters to trigger the hash function. After the transaction is verified by the blockchain network nodes, it is packaged into a new block. According to the PoS mechanism, the validator node holding and staking enough tokens completes the block production, which consumes far less energy than the traditional Proof-of-Work (PoW). The successful transaction event is permanently recorded on the chain.

[0022] Simultaneously, the storage module uploads the final PDF file to IPFS (InterPlanetary File System). IPFS uses content addressing; the file content is hashed to obtain a unique CID (Content Identifier), such as QmXyz...123. This CID is recorded in the metadata and can be uploaded to the blockchain along with the file hash. The file itself is distributed and stored across multiple nodes in the IPFS network, rather than on a single central server. (4) When it is necessary to verify the authenticity of a file, the user submits the file ID through the system interface. The verification module first queries the "file on-chain smart contract" on the blockchain through the blockchain interface module to obtain the hash value of the initially stored file. Then, it pulls the corresponding digital file from the IPFS network using the CID. The same SHA-256 algorithm is used locally to recalculate the hash of the pulled file to obtain the locally recalculated hash value. The hash value of the initially stored file is compared with the locally recalculated hash value. If they are completely consistent, it proves that the file has not been tampered with since it was digitized. If they are inconsistent, an alarm is issued immediately.

[0023] The audit trail module records the operation logs of all the above steps throughout the process, and periodically uploads the aggregated hash value of these logs to the blockchain (every 100 logs) to form a nested audit chain, ensuring that the operation history itself cannot be tampered with.

[0024] (5) Throughout the process, the user access management module uses the RBAC (Role-Based Access Control) algorithm for control. For example, when a user with the role of "file entry clerk" attempts to perform the "file upload" operation, the blockchain interface module will first call the access verification smart contract. This contract checks whether the user address has the "file entry clerk" role. If not, the transaction will be rejected. The distribution and management of keys are also implemented through smart contracts to ensure that only users with the role of "auditor" can obtain the decryption key and access the encrypted files stored on IPFS.

[0025] In this embodiment, the SHA-256 hash algorithm is a cryptographically secure hash function that maps input data of any length to a unique output of a fixed length (256 bits). This algorithm makes it extremely difficult to find two different files with the same SHA-256 hash value, thus ensuring data integrity.

[0026] In this embodiment, the CNN model used in this invention is a deep neural network that automatically learns image features through multiple layers of convolutional kernels. For an input text image, it successively passes through convolutional layers, activation layers (such as ReLU), and pooling layers, gradually extracting hierarchical features from the original pixels, from edges and contours to complex character patterns. Finally, it outputs the recognition result of each character through a fully connected layer. In this invention, the model has been specifically optimized for common archival fonts such as Song and Kai, as well as slight blurring and stains in historical documents, thus improving the recognition accuracy.

[0027] In this embodiment, Proof-of-Stake (PoS) grants validators the right to create new blocks by staking their own cryptocurrency. The probability of a validator being selected is usually proportional to the amount and duration of their staking. This mechanism securely maintains the consistency and immutability of the blockchain network while reducing energy consumption by more than 99%, making it more in line with the requirements of green IT.

[0028] In this embodiment, the Role-Based Access Control (RBAC) algorithm is a mature permission management model. In this system, the administrator predefines roles (such as "data entry clerk", "auditor", "read-only user") in the smart contract and assigns precise operation permissions to each role (such as "can call the registerHash function" and "can obtain the decryption key"). Users are assigned one or more roles, thereby automatically inheriting the corresponding permissions, realizing flexible, standardized and easy-to-manage access control.

[0029] This invention provides Embodiment 1: This experiment obtained 1,000 simulated historical documents and contracts from the public domain, including printed and handwritten text, simulating varying degrees of aging, stains, and folding marks.

[0030] This experiment was divided into three groups. The first group is an embodiment of the present invention. The second group is Comparative Example 1, which uses a traditional centralized database (such as MySQL) to store digital archives. The files are stored on a centralized file server and access is managed using username / password. The third group is Comparative Example 2, which uses a simple blockchain notarization system, that is, only the file hash is uploaded to the chain, but the original file is still stored on a centralized server and does not have a complete automated process and advanced OCR capabilities.

[0031] This experiment tested the data integrity of three groups. First, test files were stored in the three systems. After 24 hours, an attempt was made to maliciously tamper with the stored digital files (such as modifying the contract amount). Then, a verification request was initiated. The specific results are shown in the table below.

[0032]

[0033] As shown in the table above, both Example 1 and Comparative Example 2 achieved a 100% tamper detection success rate. This is because both systems can immediately identify any minor tampering and trigger an alarm by comparing the original file hash stored on the blockchain with the locally recalculated hash value. However, Comparative Example 1, due to its reliance on periodic backup comparisons rather than a real-time verification mechanism, resulted in most direct file tampering not being detected in time, with a tamper detection success rate of only 15%.

[0034] This invention provides Embodiment 2: This embodiment continues the three groups of objects in Embodiment 1 above. This experiment tests the security and performance of the three groups. By simulating the retrieval operation under high concurrency access (100 users requesting at the same time), the administrator attempts to illegally escalate his own privileges to access sensitive files, and the OCR accuracy is evaluated using the test set. The specific results are shown in the table below.

[0035]

[0036] As shown in the table above, the embodiment that executes RBAC through a blockchain smart contract achieves decentralized and automated permission verification, thereby eliminating the risk of internal super administrators abusing their privileges, and performs the best. Comparative Example 1's centralized architecture has the lowest latency under low concurrency, but poor stability under high concurrency and DDoS attacks. The embodiment utilizes the distributed characteristics of IPFS to achieve load balancing, resulting in stable and acceptable response times. Comparative Example 2, due to the need to interact with both the blockchain and the central server simultaneously, has the highest latency. The optimized CNN model used in this invention outperforms conventional commercial OCR software in accuracy for archival image recognition tasks.

[0037] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A blockchain-based end-to-end management system for digitized archives, characterized in that: Including: The document input module is used to receive physical documents and convert them into raw digital files through a scanning device, including a high-resolution scanner or a digital camera. The digitization processing module, connected to the file input module, is used to preprocess the original digital files, including image enhancement, noise removal and format standardization, and to extract text content through an OCR algorithm, which is based on a convolutional neural network for character recognition and semantic analysis. The hash calculation module, connected to the digitization processing module, is used to calculate the hash value of the preprocessed digital file using the SHA-256 hash algorithm and generate a unique digital fingerprint. The metadata management module is used to generate and store the metadata of archives, including archive number, creation time, archive type, personnel information, and digitization parameters. This metadata is associated with the digital file. The blockchain interface module, connected to the hash calculation module and the metadata management module, is used to upload hash values ​​and metadata to the blockchain network. This blockchain network is based on the Ethereum platform, adopts the proof-of-stake consensus mechanism, and runs smart contracts to execute data uploading and verification logic. A storage module for storing digital files and their metadata includes a distributed storage system based on IPFS, where digital files are stored in IPFS and IPFS hash pointers are stored in the blockchain; The verification module, connected to the blockchain interface module and the storage module, is used to obtain the stored hash value by querying the blockchain network and compare it with the locally recalculated hash value to verify the integrity and authenticity of the digital file. The audit trail module is used to record the operation log of the entire process of document digitization, including scanning time, processing steps and on-chain time, and uploads the log hash value to the blockchain through the blockchain interface module to ensure that the operation is tamper-proof; The user access management module, based on a role-based access control algorithm, manages the access permissions of different users to the system, including data entry clerks, auditors, and administrators, and uses smart contracts to implement the allocation and verification of permissions.

2. The blockchain-based end-to-end management system for digitizing archives according to claim 1, characterized in that: The digitization module also includes a quality detection unit, which is used to automatically detect the quality indicators of digital files based on image processing algorithms, including resolution, sharpness and color fidelity. The image processing algorithm uses the Sobel operator for edge detection and PSNR calculation. If the quality does not meet the standards, a re-digitization process is triggered.

3. The blockchain-based end-to-end management system for digitizing archives according to claim 1, characterized in that: The smart contracts in the blockchain interface module include an archive upload contract and a verification contract. The archive upload contract is used to receive hash values ​​and metadata, package them into transactions, and submit them to the blockchain. The verification contract is used to respond to external queries and return hash value comparison results. The smart contract includes an event triggering mechanism that automatically notifies relevant users when data upload or verification fails.

4. The blockchain-based end-to-end management system for digitizing archives according to claim 1, characterized in that: The distributed storage system of the storage module uses IPFS and combines it with an encryption algorithm to encrypt digital files. The encryption algorithm is based on AES-256 symmetric encryption, and the key is managed through a blockchain smart contract. Only authorized users can obtain the decryption key through the permission management module.

5. The blockchain-based end-to-end management system for digitized archives as described in claim 1, characterized in that: The verification module also includes a timestamp verification unit, which is used to verify the time sequence of the digitized archives through the timestamp service in the blockchain network. The timestamp service is based on the blockchain block generation time and synchronized with NTP to ensure that the operation time cannot be tampered with.

6. A blockchain-based end-to-end management method for digitized archives, employing the blockchain-based end-to-end management system for digitized archives as described in any one of claims 1-5, characterized in that: Includes the following steps: S1 receives physical files through the file input module and converts them into raw digital files using a scanning device; S2 preprocesses the original digital file through the digitization processing module, including image enhancement, noise removal and format standardization, and extracts the text content through the OCR algorithm, which is based on CNN for character recognition; S3 uses the hash calculation module to calculate the hash value of the preprocessed digital file using the SHA-256 algorithm, generating a unique digital fingerprint; S4 generates metadata for archives through the metadata management module, including archive number, creation time, and personnel information, and associates the metadata with digital files; S5 uploads hash values ​​and metadata to the blockchain network through the blockchain interface module. This blockchain network is based on the Ethereum platform, adopts the PoS consensus mechanism, and automatically executes data uploading to the chain through smart contracts. S6 stores digital files into the distributed storage system IPFS through the storage module and stores IPFS hash pointers in the blockchain; S7: The verification module queries the blockchain network to obtain the stored hash value and compares it with the locally recalculated hash value to verify the integrity and authenticity of the digital file. S8 records the entire process operation log through the audit trail module and puts the log hash value on the blockchain to ensure that the operation cannot be tampered with. S9 manages user access permissions based on the RBAC algorithm through the user permission management module, and implements automatic permission verification through smart contracts.

7. The blockchain-based end-to-end management method for digital archives according to claim 6, characterized in that: In step S2, the preprocessing also includes a quality detection step, which automatically detects the quality indicators of the digital file based on the image processing algorithm, including resolution, sharpness and color fidelity. The image processing algorithm uses the Sobel operator for edge detection and PSNR calculation. If the quality does not meet the standard, the re-digitization process is triggered.

8. The blockchain-based end-to-end management method for digital archives according to claim 6, characterized in that: In step S5, the data upload is executed through a smart contract, which includes an archive upload contract and a verification contract. The archive upload contract is used to receive hash values ​​and metadata, package them into a transaction, and submit them to the blockchain. The verification contract is used to respond to external queries and return hash value comparison results.

9. The blockchain-based end-to-end management method for digital archives according to claim 6, characterized in that: In step S6, storing the digital file includes an encryption step, where the digital file is encrypted using the AES-256 symmetric encryption algorithm. The key is managed through a blockchain smart contract, and only authorized users can obtain the decryption key through the permission management module.

10. The blockchain-based end-to-end management method for digital archives according to claim 6, characterized in that: In step S7, the verification also includes a timestamp verification step, which verifies the time sequence of the digitized archive through the timestamp service in the blockchain network. The timestamp service is based on the blockchain block generation time and synchronized with NTP to ensure that the operation time cannot be tampered with.

Images