Security processing data solidification method and system based on mass data

Abstract

The invention relates to a security processing data solidification method and system based on mass data, and belongs to the technical field of data storage processing. The method comprises the following steps: acquiring to-be-stored data, and performing encryption according to the to-be-stored data through an improved AES-ECC hybrid encryption algorithm to obtain to-be-stored encrypted data; constructing a block chain network, and storing the data to block chain nodes through a PBFT consensus algorithm according to the to-be-stored encrypted data; the stored encrypted data is backed up to a plurality of block chain nodes, and curing processing is performed through the data snapshot, so that strong encryption protection in the data storage and transmission process is realized, data leakage and unauthorized access are effectively prevented, data integrity is ensured, and data traceability is realized; meanwhile, data loss prevention is achieved, recovery can be carried out when a system fault or data damage occurs, the block chain network can be expanded according to needs, and the storage and processing requirements of a large amount of data are met.

Description

Technical Field

[0001] The present invention belongs to the technical field of data storage processing, and in particular to a method and system for securely processing data solidification based on a large amount of data. Background Art

[0002] In the era of big data, data has become an extremely important asset for enterprises, governments and individuals. The explosive growth of data has brought about a series of data security issues. How to efficiently and securely process and analyze large amounts of data to ensure data privacy and data integrity is an issue that needs to be addressed urgently.

[0003] Existing data security processing methods have the following shortcomings: when processing sensitive data, existing technologies may not fully guarantee the privacy of data, and data is vulnerable to unauthorized access, tampering or destruction. At the same time, during data transmission and storage, data is at risk of leakage due to system vulnerabilities or illegal intrusions, while factors such as hardware failures and network problems may cause data to be damaged during transmission or processing, affecting data integrity. In addition, traditional data processing methods are often unable to effectively process massive amounts of data, and data management efficiency is low.

[0004] Data solidification refers to saving data in a fixed form to ensure that the data will not be easily modified or tampered with after being saved, thereby ensuring the integrity and reliability of the data. It is usually used in scenarios where it is necessary to ensure that the data cannot be changed, such as legal documents, audit records, electronic contracts, etc., and can effectively improve the security and reliability of data. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention proposes a data solidification method and system for securely processing a large amount of data.

[0006] The purpose of the present invention can be achieved through the following technical solutions: A data solidification method for secure processing of a large amount of data, comprising: Acquire the data to be stored, and encrypt the data to be stored by using an improved AES-ECC hybrid encryption algorithm to obtain the encrypted data to be stored; Construct a blockchain network, and store the data on the blockchain node through the PBFT consensus algorithm according to the encrypted data to be stored; The stored encrypted data is backed up to multiple blockchain nodes and solidified through data snapshots.

[0007] Preferably, the obtaining of the data to be stored, and encrypting the data to be stored by using an improved AES-ECC hybrid encryption algorithm to obtain the encrypted data to be stored comprises: Setting initial parameters of the Logistic map, the initial parameters including but not limited to bifurcation parameters and initial values; Generate chaotic sequence and initialize AES key through Logistic mapping; Generate a chaotic sequence through Logistic mapping and initialize an ECC private key, and calculate a corresponding ECC public key according to the ECC private key; Preprocessing the data to be stored to obtain standard data to be stored, wherein the preprocessing includes but is not limited to padding and segmentation; Encrypting the standard data to be stored by using the AES key to obtain AES encrypted data; Encrypting the AES key by using the ECC public key to obtain an ECC enhanced AES key; The AES encrypted data and the ECC enhanced AES key are combined to obtain the encrypted data to be stored.

[0008] Preferably, the bifurcation parameter is used to control the behavior of the Logistic mapping, and the initial value is the starting point of the iterative process of the Logistic mapping generating the chaotic sequence.

[0009] Preferably, the building of a blockchain network includes: Preset the number of nodes participating in the consensus and deploy blockchain nodes on virtual servers; The unique identifier and network parameters of the blockchain node are configured, and a point-to-point connection is established through encrypted communication. The unique identifier is used to distinguish different nodes in the blockchain network, and the network parameters include but are not limited to IP address and port number.

[0010] Preferably, establishing a point-to-point connection through encrypted communication includes exchanging digital certificates between nodes for verifying the identity of the nodes. When establishing the connection, the nodes negotiate a session key for encrypting subsequent communications through the TLS / SSL protocol. Once the encrypted connection is established, the nodes can securely transmit transaction information, block data and consensus messages.

[0011] Preferably, storing data on a blockchain node through a PBFT consensus algorithm according to the encrypted data to be stored includes: The blockchain master node receives the transaction request from the client, verifies the validity of the transaction request and packages the verified transaction requests into blocks, generates a pre-prepared message and broadcasts it to other blockchain replica nodes. The transaction request includes but is not limited to the encrypted data to be stored, and the pre-prepared message includes the hash of the block and other necessary information; After receiving the pre-prepare message, the current blockchain replica node verifies the validity of the transaction request, generates a prepare message after the verification is passed, and broadcasts it to other blockchain replica nodes; When a blockchain replica node receives 2f+1 prepare messages from different replica nodes, it broadcasts a confirmation message to indicate that it is ready to submit the block to the blockchain; when all blockchain replica nodes have received at least 2f+1 confirmation messages, the block reaches a consensus and adds the block to the blockchain copy of each node, where f is the number of Byzantine nodes that can be tolerated.

[0012] Preferably, the step of backing up the stored encrypted data to multiple blockchain nodes and solidifying the data through data snapshots includes: Select multiple reliable blockchain nodes in the network as backup nodes, where the reliable blockchain nodes have stable network connections and sufficient storage space; On the blockchain master node, a data snapshot of the current blockchain state is taken and the created data snapshot is encrypted and transmitted to the backup node; On the backup node, decrypt the received data snapshot and verify the integrity of the data snapshot to ensure consistency with the data of the primary node; If the data snapshot integrity verification fails, retransmitting the data snapshot; If the data snapshot integrity verification succeeds, the data snapshot is solidified into a storage medium on the backup node, and the storage medium includes but is not limited to a hard disk, an SSD, and a dedicated backup storage system.

[0013] A data solidification system for secure processing of large amounts of data, applied to the above-mentioned data solidification method for secure processing of large amounts of data, comprising a data encryption processing module, a data storage module, and a data backup and solidification module; The data encryption processing module is used to obtain the data to be stored, and encrypt the data to be stored by using an improved AES-ECC hybrid encryption algorithm to obtain the encrypted data to be stored; The data storage module is used to build a blockchain network, and stores data on the blockchain node through the PBFT consensus algorithm according to the encrypted data to be stored; The data backup and solidification module is used to back up the stored encrypted data to multiple blockchain nodes and solidify the data through data snapshots.

[0014] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned data solidification method for secure processing based on a large amount of data when executing the computer program.

[0015] A storage medium containing computer executable instructions, which are used to execute the above-mentioned data solidification method for secure processing based on large amounts of data when executed by a computer processor.

[0016] The beneficial effects of the present invention are: (1) By improving the AES-ECC hybrid encryption algorithm, data is strongly encrypted during storage and transmission, effectively preventing data leakage and unauthorized access. The AES algorithm performs high-efficiency data encryption, encrypts the AES key with the ECC public key and retains the ECC private key. The combination of the two improves the security and efficiency of data storage, solves the problems of poor security and easy leakage of symmetric encryption technology AES and the large amount of calculation and low efficiency of asymmetric encryption technology ECC. The AES-ECC hybrid encryption algorithm is improved by Logistic mapping, which improves the randomness and complexity of the key.

[0017] (2) Data is stored on multiple nodes of the blockchain network through distributed storage. Even if some nodes fail, other nodes can still provide data, which improves data availability. At the same time, the immutability of blockchain is used to ensure that the stored data will not be maliciously modified, ensuring the integrity of the data. The chain structure of blockchain is used to track the source of data and the history of changes, which is convenient for auditing and supervision, and realizes data traceability. (3) Ensure that all nodes reach consensus on data consistency through the PBFT consensus algorithm. The PBFT algorithm tolerates a certain number of Byzantine nodes, that is, nodes that may exhibit any malicious behavior, to ensure that the blockchain network can continue to operate normally even when some nodes have problems; (4) By setting up a backup mechanism, encrypted data can be backed up to multiple blockchain nodes to prevent data loss. Data snapshots can be used to solidify the data status at a specific point in time, making it easier to recover in the event of system failure or data corruption. At the same time, the blockchain network can be expanded as needed to adapt to the storage and processing needs of large amounts of data. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.

[0019] Figure 1 The present invention is a flowchart of a method for securely processing data solidification based on large amounts of data. DETAILED DESCRIPTION

[0020] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects of the present invention are described in detail below in conjunction with the accompanying drawings and preferred embodiments.

[0021] See also Figure 1 , a data solidification method for secure processing of a large amount of data, comprising: S1: Acquire data to be stored, and encrypt the data to be stored by using an improved AES-ECC hybrid encryption algorithm to obtain encrypted data to be stored; S2: Building a blockchain network, storing data on blockchain nodes through the PBFT consensus algorithm according to the encrypted data to be stored; S3: Back up the stored encrypted data to multiple blockchain nodes and solidify it through data snapshots.

[0022] In this embodiment, the obtaining of the data to be stored and encrypting the data to be stored by using an improved AES-ECC hybrid encryption algorithm to obtain the encrypted data to be stored are specifically implemented by the following steps: S101: Setting initial parameters of the Logistic map, wherein the initial parameters include but are not limited to bifurcation parameters and initial values; S102: Generate a chaotic sequence through Logistic mapping and initialize the AES key; S103: Generate a chaotic sequence through Logistic mapping and initialize an ECC private key, and calculate a corresponding ECC public key according to the ECC private key; Specifically, the ECC public key corresponding to the ECC private key is calculated by scalar multiplication of the elliptic curve, and the ECC public key calculation is implemented using the ecdsa library.

[0023] S104: preprocessing the data to be stored to obtain standard data to be stored, including but not limited to filling and segmentation; It should be noted that the segmentation is used to divide the data into smaller fixed-length data blocks for storage or processing, and the padding is used to ensure that the data blocks meet specific length requirements.

[0024] S105: Encrypting the standard data to be stored by using the AES key to obtain AES encrypted data; S106: Encrypt the AES key using the ECC public key to obtain an ECC enhanced AES key; S107: Combining the AES encrypted data with the ECC enhanced AES key to obtain encrypted data to be stored.

[0025] It should be noted that the Logistic map is a classic chaotic dynamics model, and the bifurcation parameter is used to control the behavior of the Logistic map. Under different bifurcation parameter values, the Logistic map can exhibit different dynamic behaviors such as periodicity, chaos and fixed points. The initial value is the starting point of the iterative process of the Logistic map generating a chaotic sequence, which is usually selected between 0 and 1; the generating a chaotic sequence through the Logistic map and initializing the key is to use the Logistic map to iteratively generate a chaotic sequence, and extract a sufficient number of bits from the chaotic sequence to form a key.

[0026] In this embodiment, the corresponding data decryption process is to extract the chaotic sequence from the received encrypted data packet, use the recipient's ECC private key to decrypt the ECC enhanced AES key to obtain the AES key, and decrypt the data according to the AES key to obtain the original data before encryption.

[0027] In this embodiment, the construction of the blockchain network is specifically implemented by the following steps: storing the data on the blockchain node through the PBFT consensus algorithm according to the encrypted data to be stored: S201: Preset the number of nodes participating in the consensus and deploy blockchain nodes on the virtual server; S202: configuring a unique identifier and network parameters of the blockchain node, and establishing a point-to-point connection through encrypted communication, wherein the unique identifier is used to distinguish different nodes in the blockchain network, and the network parameters include but are not limited to an IP address and a port number; It should be noted that establishing a point-to-point connection through encrypted communication includes exchanging digital certificates between nodes for verifying the identity of the nodes. When establishing the connection, the nodes negotiate a session key for encrypting subsequent communications through the TLS / SSL protocol. Once the encrypted connection is established, the nodes can securely transmit transaction information, block data and consensus messages.

[0028] S203: The blockchain master node receives the transaction request from the client, verifies the validity of the transaction request and packages the verified transaction requests into blocks, generates a pre-prepared message and broadcasts it to other blockchain replica nodes, wherein the transaction request includes but is not limited to the encrypted data to be stored, and the pre-prepared message includes the hash of the block and other necessary information; S204: After receiving the pre-prepare message, the current blockchain replica node verifies the validity of the transaction request, generates a prepare message after the verification is passed, and broadcasts it to other blockchain replica nodes; S205: When the blockchain replica node receives 2f+1 prepare messages from different replica nodes, it broadcasts a confirmation message to indicate that it is ready to submit the block to the blockchain; when all blockchain replica nodes have received at least 2f+1 confirmation messages, the block reaches a consensus and adds the block to the blockchain copy of each node.

[0029] Specifically, the nodes in the blockchain network reach a consensus through the PBFT consensus algorithm, confirm the validity of the block, and add the consensus block to the end of the blockchain to form a chain structure. At this time, the encrypted data to be stored is formally recorded on the blockchain.

[0030] It should be noted that, in the PBFT consensus algorithm, f is the number of Byzantine nodes that can be tolerated.

[0031] In this embodiment, the stored encrypted data is backed up to multiple blockchain nodes and solidified through data snapshots through the following steps: S301: Select multiple reliable blockchain nodes in the network as backup nodes, where the reliable blockchain nodes are blockchain nodes with stable network connections and sufficient storage space; S302: On the blockchain master node, a data snapshot of the current blockchain state is taken and the created data snapshot is encrypted and transmitted to the backup node; S303: On the backup node, decrypt the received data snapshot and verify the integrity of the data snapshot to ensure consistency with the data of the primary node; If the data snapshot integrity verification fails, retransmitting the data snapshot; If the data snapshot integrity verification succeeds, the data snapshot is solidified into a storage medium on the backup node, and the storage medium includes but is not limited to a hard disk, an SSD, and a dedicated backup storage system.

[0032] It should be noted that in this embodiment, the solidified data needs to be marked to record information such as the solidification time and snapshot version to facilitate data management and data recovery. Data snapshots and backup processes are performed regularly to keep the data up to date and the status of backup nodes is monitored regularly to ensure the availability and reliability of the backup. At the same time, backup and recovery tests are performed regularly to ensure that data can be quickly restored when needed.

[0033] A data solidification system based on the secure processing of large amounts of data, comprising a data encryption processing module, a data storage module, and a data backup and solidification module; The data encryption processing module is used to obtain the data to be stored, and encrypt the data to be stored by using an improved AES-ECC hybrid encryption algorithm to obtain the encrypted data to be stored; The data storage module is used to construct a blockchain network, and stores data on blockchain nodes through an improved consensus algorithm according to the encrypted data to be stored; The data backup and solidification module is used to back up the stored encrypted data to multiple blockchain nodes and solidify the data through data snapshots.

[0034] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.

[0035] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0036] The program code included on the computer readable medium can be transmitted with any appropriate medium, including but not limited to wireless, electric wire, optical cable, RF, etc., or any suitable combination of the above. The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or their combinations, and the programming language includes object-oriented programming languages-such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0037] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A data solidification method for secure processing of large amounts of data, characterized in that: include: Acquire the data to be stored, and encrypt the data to be stored by using an improved AES-ECC hybrid encryption algorithm to obtain the encrypted data to be stored; Construct a blockchain network, and store the data on the blockchain node through the PBFT consensus algorithm according to the encrypted data to be stored; The stored encrypted data is backed up to multiple blockchain nodes and solidified through data snapshots.

2. The method for securely processing data based on a large amount of data according to claim 1, characterized in that: The obtaining of the data to be stored, and encrypting the data to be stored by using an improved AES-ECC hybrid encryption algorithm to obtain the encrypted data to be stored comprises: Setting initial parameters of the Logistic map, the initial parameters including but not limited to bifurcation parameters and initial values; Generate chaotic sequence and initialize AES key through Logistic mapping; Generate a chaotic sequence through Logistic mapping and initialize an ECC private key, and calculate a corresponding ECC public key according to the ECC private key; Preprocessing the data to be stored to obtain standard data to be stored, wherein the preprocessing includes but is not limited to padding and segmentation; Encrypting the standard data to be stored by using the AES key to obtain AES encrypted data; Encrypting the AES key by using the ECC public key to obtain an ECC enhanced AES key; The AES encrypted data and the ECC enhanced AES key are combined to obtain the encrypted data to be stored.

3. The method for securely processing data based on a large amount of data according to claim 2, characterized in that: The bifurcation parameter is used to control the behavior of the Logistic map, and the initial value is the starting point of the iterative process of the Logistic map generating a chaotic sequence.

4. The method for securely processing data based on a large amount of data according to claim 1, characterized in that: The construction of the blockchain network includes: Preset the number of nodes participating in the consensus and deploy blockchain nodes on virtual servers; The unique identifier and network parameters of the blockchain node are configured, and a point-to-point connection is established through encrypted communication. The unique identifier is used to distinguish different nodes in the blockchain network, and the network parameters include but are not limited to IP address and port number.

5. The method for securely processing data based on a large amount of data according to claim 4, characterized in that: The establishment of a point-to-point connection through encrypted communication includes exchanging digital certificates between nodes for verifying the identity of the nodes. When establishing a connection, the nodes negotiate a session key for encrypting subsequent communications through the TLS / SSL protocol. Once the encrypted connection is established, the nodes can securely transmit transaction information, block data and consensus messages.

6. The method for securely processing data based on a large amount of data according to claim 1, characterized in that: The storing of data on the blockchain node through the PBFT consensus algorithm according to the encrypted data to be stored includes: The blockchain master node receives the transaction request from the client, verifies the validity of the transaction request and packages the verified transaction requests into blocks, generates a pre-prepared message and broadcasts it to other blockchain replica nodes. The transaction request includes but is not limited to the encrypted data to be stored, and the pre-prepared message includes the hash of the block and other necessary information; After receiving the pre-prepare message, the current blockchain replica node verifies the validity of the transaction request, generates a prepare message after the verification is passed, and broadcasts it to other blockchain replica nodes; When a blockchain replica node receives 2f+1 prepare messages from different replica nodes, it broadcasts a confirmation message to indicate that it is ready to submit the block to the blockchain; when all blockchain replica nodes have received at least 2f+1 confirmation messages, the block reaches a consensus and adds the block to the blockchain copy of each node, where f is the number of Byzantine nodes that can be tolerated.

7. The method for securely processing data based on a large amount of data according to claim 1, characterized in that: The backing up of the stored encrypted data to multiple blockchain nodes and solidifying the data through data snapshots includes: Select multiple reliable blockchain nodes in the network as backup nodes, where the reliable blockchain nodes have stable network connections and sufficient storage space; On the blockchain master node, a data snapshot of the current blockchain state is taken and the created data snapshot is encrypted and transmitted to the backup node; On the backup node, decrypt the received data snapshot and verify the integrity of the data snapshot to ensure consistency with the data of the primary node; If the data snapshot integrity verification fails, retransmitting the data snapshot; If the data snapshot integrity verification succeeds, the data snapshot is solidified into a storage medium on the backup node, and the storage medium includes but is not limited to a hard disk, an SSD, and a dedicated backup storage system.

8. A data solidification system for secure processing of large amounts of data, applied to the data solidification method for secure processing of large amounts of data as claimed in any one of claims 1 to 7, characterized in that: It includes data encryption processing module, data storage module, data backup and curing module; The data encryption processing module is used to obtain the data to be stored, and encrypt the data to be stored by using an improved AES-ECC hybrid encryption algorithm to obtain the encrypted data to be stored; The data storage module is used to build a blockchain network, and stores data on the blockchain node through the PBFT consensus algorithm according to the encrypted data to be stored; The data backup and solidification module is used to back up the stored encrypted data to multiple blockchain nodes and solidify the data through data snapshots.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for securely processing data solidification based on a large amount of data as described in any one of claims 1-7 is implemented.

10. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the data solidification method based on secure processing of large amounts of data as described in any one of claims 1-7 when executed by a computer processor.

Images