An easily expandable and traceable data operation recording system and method

Abstract

The present invention discloses an easily scalable and traceable data operation recording system and method, based on an easily scalable blockchain architecture. The architecture includes modules such as a sending client, a query client, an access server, a blockchain network, a blockchain management server, and a client, as well as interaction schemes between them. The present invention discloses an easily scalable and traceable data operation recording system and method, which can save the entire content of the operation record on the chain, providing tamper-proof and anti-destruction effects. When the amount of data stored in real time increases, or the blockchain storage efficiency decreases, or the blockchain node disk space is insufficient, the blockchain network can be automatically expanded or notified to the administrator to meet data storage needs. A data encoding scheme and corresponding storage and query schemes can be used to reduce the amount of data.

Description

Technical Field

[0001] The present invention relates to an easily expandable and traceable data operation recording system and method, belonging to the technical field of data operation recording. Background Art

[0002] Traditional software information systems typically use file logs or database operation records to record and track user behavior. User operation records often contain important user information, and if malicious data manipulation occurs, the responsible party can be determined based on the user operation records. If operation records are stored in logs, they are large and cannot be retained for long periods of time. If records are stored in databases, they also pose the risk of tampering.

[0003] Some existing methods utilize the decentralized and tamper-resistant nature of blockchain to write operation records or logs into the blockchain. When problems arise, they can be used to trace back to the blockchain. However, blockchain technology also has some problems. 1. Low efficiency of the consensus algorithm: Because the traditional blockchain consensus mechanism requires consensus to be reached in a collaborative environment, communication complexity is high, and system transaction performance decreases significantly with the increase in the scale of network nodes, resulting in low storage efficiency. 2. It is not suitable for storing large amounts of data. Because the blockchain structure cannot be tampered with, it generally does not support deleting some of the data. Each peer node must store a complete copy of the on-chain data. If the transaction content is large, it will easily cause the hard drive to fill up quickly, resulting in lower transmission and storage efficiency and slower chain upload speed.

[0004] Existing technology uploads operation records to the blockchain in JSON format. JSON format contains multiple key-value pairs, all in string format, resulting in a large amount of data. Therefore, the common solution is to store the data off-chain and store the data digest (hash value) on-chain. However, existing technology has the following shortcomings:

[0005] 1. Uploading operation records to the blockchain in JSON format. JSON format contains multiple key-value pairs, all in string format, and the data volume is large. As time goes by, the hard disk will easily fill up, and no corresponding solution is provided.

[0006] 2. No solution is proposed to the problem of low blockchain storage efficiency.

[0007] 3. Only log summary data is stored on the chain, which can only verify whether the log data has been tampered with. If the log data is damaged, the content of the previous operation cannot be known.

[0008] 4. Operation records or logs become invalid after a period of time. To reduce storage space, blocks are periodically deleted to reduce data volume. However, deleting blocks reduces the blockchain's tamper-resistance. Therefore, most existing open-source blockchain platforms do not support block deletion, as it can cause platform issues. Some blockchains also use Merkle trees, constructing a Merkle tree using the digest of each block to ensure the blockchain has not been tampered with. Directly deleting blocks renders this verification method ineffective.

[0009] 5. No solution is proposed to the problem of low blockchain storage efficiency.

[0010] Therefore, how to combine data operation records with blockchain and solve the shortcomings of existing technologies is a technical problem that technicians in this field urgently need to solve. Summary of the Invention

[0011] Objective: To overcome the deficiencies in the prior art, the present invention provides an easily scalable and traceable data operation recording system and method.

[0012] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:

[0013] An easily expandable and traceable data operation recording system, including: a sending client, a query client, an access server, a blockchain network, a blockchain management server, and a blockchain management client.

[0014] The access server receives the operation records sent by the sending client, selects a blockchain network from all blockchain networks according to the sending strategy, and sends the operation records to the blockchain network for storage; accepts the query request issued by the query client, and selects one or more blockchain networks for query according to the query conditions, and merges and returns the query operation record list in the order of storage; the access server manages which sending clients and query clients can access it according to the configured access rules.

[0015] The blockchain management server manages the blockchain network through the blockchain management client, including but not limited to installation, deployment, configuration, maintenance, and deletion operations, and synchronizes blockchain network information and access credentials to the access server.

[0016] The access server or the blockchain management client sends an alarm to the blockchain management server, and the blockchain management server automatically or notifies the administrator to deploy a new blockchain network according to the configuration.

[0017] As a preferred solution, the data operation record system also includes multiple access servers. When the blockchain is a consortium chain, it supports collaboration among multiple organizations. Each access server corresponds to one of the organizations. Each organization customizes access policies based on its needs and manages the list of information systems accessible to the organization. When the blockchain is a private chain, only one access server is deployed. Furthermore, when the blockchain is a private chain, multiple access servers can be deployed depending on the usage scenario.

[0018] As a preferred solution, the access server includes: an access control module, a storage module, a query module and a blockchain interaction module.

[0019] The access control module is used to manage which information systems can access the access server.

[0020] The storage module accepts a request from an external information system to store operation records, and sends the operation records to the blockchain network node for storage through the blockchain interaction module.

[0021] The query module accepts the query request issued by the query client, selects one or more blockchain networks according to the query conditions, queries the corresponding nodes in each network through the blockchain interaction module, and merges and returns the query operation record list in the order of storage.

[0022] As a preferred solution, the access control module also includes a list of accessible users and the permissions of each user, each user can have one or more permissions. It also includes a CA, from which users apply for certificates, and the CA is responsible for issuing private keys to users.

[0023] As a preferred solution, the user's authority is achieved by adding the user's public key to the corresponding authority list.

[0024] As a preferred solution, the storage module of the access server includes four submodules: a receiving module, a policy module, a sending module, and an early warning module.

[0025] The receiving module accepts the storage request from the external information system, puts the message into the cache, and optionally adds a sequence tag to the message.

[0026] The sending module reads the first operation record data entered from the cache, selects a storable blockchain network from the blockchain network list according to the sending strategy formulated by the policy module, obtains accessible node information, calls the blockchain interaction module according to the node information, and the blockchain interaction module sends the operation record to the blockchain node for storage.

[0027] The policies in the policy module are configured by an administrator, and the policies include but are not limited to: a polling method, and are selected according to the configured priority.

[0028] The early warning module monitors the cache area and sends an alarm to the blockchain management server when it finds that the amount of data exceeds the threshold.

[0029] As a preferred solution, the blockchain network list includes but is not limited to the blockchain network ID, start time, end time, and status fields. The blockchain network ID is the unique identifier of the blockchain network. The start time and end time respectively identify the earliest and latest data times stored by the blockchain network. The status field is used to indicate whether the blockchain network is available for storage. The node information of the blockchain network includes but is not limited to: the blockchain network ID to which the node belongs, organization information, deployment server IP address, access port, and access credential information.

[0030] As a preferred solution, the query module of the access server includes four submodules: an accepting module, a query module, an integrating module, and a returning module.

[0031] The receiving module accepts query requests from external information systems.

[0032] Based on the time period in the query request, the query module selects a list of blockchain networks containing data for that time period from all blockchain networks. Based on this list, the query module retrieves node information for the blockchain networks accessible to the access server to which the query module belongs. The query request and the node information for the blockchain networks in the list are then sent to the blockchain interaction module, which then forwards the query request to the blockchain nodes for querying. The query results are then passed to the integration module for integration.

[0033] The integration module sorts the returned operation record list by timestamp. If the timestamps are the same and the data is in the same blockchain network, it is sorted according to the original order returned. If the timestamps are the same and the data is in different blockchain networks, if there is a sequence tag, it is sorted according to the sequence tag. The sorted operation record list is handed over to the return module and returned to the external information system.

[0034] As a preferred solution, the blockchain interaction module of the access server manages the identity credentials for blockchain node interaction, namely the private key issued by the blockchain network CA. It sends data to the blockchain network node to submit transactions, and queries the corresponding data transaction information list from the blockchain network node according to the query conditions.

[0035] As a preferred solution, the blockchain management server includes three sub-modules: a deployment module, a deletion module, and a configuration management module, among which the deletion module is an optional module.

[0036] The deployment module sends a deployment instruction to the blockchain management client on the server.

[0037] The deletion module is used by the administrator to delete the invalid blockchain network.

[0038] The configuration management module is used to manage the organizational structure information of the blockchain network, as well as the server information provided by each organization, and also includes the access credentials of the blockchain network nodes.

[0039] As a preferred solution, the instructions sent by the deployment module include the configuration information of the blockchain network and the organizational structure information of each node. The deployment module is deployed by the administrator by calling the deployment module to provide an interface. When the blockchain management server is configured to allow automatic deployment, the deployment process is entered by receiving alarms from the access server and the blockchain management client. When the blockchain management server is configured to not allow automatic deployment, an alarm is sent to notify the administrator to manually deploy. When generating the networking information of the blockchain network, the deployment module will comprehensively generate it based on the server information provided by each organization in the configuration management module and the performance data information obtained from the blockchain management client to ensure that the new blockchain network server meets the blockchain operation requirements. The networking information is generated by the administrator; the organizational structure information is obtained from the configuration management module.

[0040] As a preferred solution, the configuration management module manages the node information of the blockchain network currently available for each organization member, as well as the node information of all accessible blockchain networks, and records the time range corresponding to the operation record data stored by each blockchain node. When the deployment module deploys a new blockchain network, it sends a message to the configuration management module to update the currently available node information of the blockchain network. The configuration management module then applies for access credentials from the corresponding node in the blockchain network for each organization through the blockchain management client and sends them to the corresponding access server of the blockchain network.

[0041] As a preferred solution, the blockchain management client runs on the server where the blockchain node is installed or to be installed. The blockchain management client includes four sub-modules: deployment module, monitoring module, deletion module, and configuration management module, among which the deletion module is an optional module.

[0042] The deployment module receives the deployment instructions sent by the deployment module of the blockchain management server, and deploys the blockchain network node on the server according to the configuration information in the instructions.

[0043] The monitoring module regularly checks the performance indicators of the server. When it finds that the disk space is insufficient or other performance indicators cannot meet the requirements, it generates an alarm message and sends it to the blockchain management server.

[0044] The deletion module receives the deletion instruction from the blockchain management server and deletes the corresponding blockchain network node according to the instruction information.

[0045] After the deployment is completed, the deployment module of the blockchain management client will also call the configuration management module, use the organization information in the deployment instruction as identity information, apply for access credentials from the blockchain network node on the server, and send it to the blockchain management server.

[0046] As a preferred solution, nodes belonging to multiple blockchain networks can be deployed on the same server.

[0047] As a preferred solution, there are multiple blockchain networks, including blockchain networks that can be continuously stored at present, and blockchain networks that cannot be used for further storage due to full disk space or other reasons. All blockchain networks can be used for query.

[0048] As a preferred solution, the query client includes: an encoding module, a sending module, a receiving module, a decoding module, an encryption module, and a decryption module, wherein the encryption module and the decryption module are optional modules.

[0049] The encoding module encodes part of the query keywords in the query request, where the part of the query keywords is the keywords that have been encoded during storage.

[0050] Optionally, when searching using ciphertext, the encoded or non-encoded query field is encrypted by the encryption module.

[0051] The sending module sends the query keyword to the access server.

[0052] The accepting module accepts the query result returned by the access server.

[0053] The optional decryption module decrypts the ciphertext portion of the operation record in the query result.

[0054] The decoding module decodes the encoded part and then returns the decoded operation record information.

[0055] As a preferred solution, the query client is integrated into the external information system as a plug-in, or independently runs on a server to receive query requests from the external information system through an interface. The server is the server where the external information system is deployed, or the server deployed by the access server.

[0056] As a preferred embodiment, the sending client is integrated into the external information system as a plug-in, or runs independently on the server where the external information system resides. When the sending client is a plug-in, it includes an encoding module, an encryption module, and a message sending module, with the encryption module being optional. The external information system transmits data to the sending client by invoking an interface provided by the sending client plug-in. The encoding module encodes part of the operation log data, and the encryption module optionally encrypts part of the operation log information. The encrypted information is either not used as a query condition, or is used as a query condition but can be queried via ciphertext. When the sending client does not invade the external information system, an interface is provided to receive messages from the external information system. The optional sending client also includes a formatting module. The formatting module can convert the operation records generated by the information system into data that can be processed by the encoding module. The formatting module is customized for the specific operation record format of the external information system; when the sending client does not invade the external information system, the operation record information is obtained by reading the log file or memory of the external information system. The optional sending client also includes a reading module. The reading module reads the operation record from the file or memory, filters the information, and then sends it to the formatting module for format conversion. The reading module is customized for the specific log file type and format of the external information system.

[0057] In a second aspect, an easily scalable and traceable data operation recording method based on the operation recording system described in the first aspect includes the following methods:

[0058] Operation record data storage method, operation record data query method, blockchain network expansion method.

[0059] The operation record data storage method comprises the following steps:

[0060] The external information system inputs the operation record data, and the sending client stores the operation record (when the sending client is a plug-in or provides a calling interface); the sending client can also read the log file or memory in the external information system to obtain the operation record data.

[0061] The operation record data storage method comprises the following steps:

[0062] The sending client processes the operation record data including:

[0063] Filtering: The filtering step is an optional operation used to filter key operation records that need to be stored, and is completed by the reading module in the sending client.

[0064] Formatting: The formatting step is an optional operation used to convert the operation record data into a format that can be encoded, and is completed by the formatting module in the sending client.

[0065] Encoding: Encode part of the data in the operation record to compress the operation record data. The specific steps are as follows:

[0066] Encode fields whose values ​​can be exhaustively enumerated (such as system, operation, operation object, etc.).

[0067] Suppose that a certain exhaustible field t has n different values, and the set of values ​​is defined as X. Now construct a binary code set Y of n numbers from 0 to n-1, and define a bijective function f from X to Y so that any value x in X can obtain a binary code y in the set Y through f(x). According to any code y in Y, the inverse function of f is denoted as f -1 , calculate f -1 (y) gets the value x. Let k be a value that satisfies 2 k If the smallest natural number ≥ n is used, k bits are used to store these n codes. Then field t uses k bits to store its corresponding code. It also includes: reserving field values ​​for actual use.

[0068] Assume that there are n fields whose values ​​can be exhaustively enumerated. After binary encoding each field i, the number of bits occupied by each field is ki. The encoding of the field can be generated by concatenating the fields. The number of bits occupied by the encoding is from k1 to k n The sum of

[0069] Encryption: The encryption step is an optional operation, which encrypts part of the operation record information. The encrypted information is information that is not used as a query condition, or information that is used as a query condition but can be queried through ciphertext.

[0070] The sending client sends the processed operation record to the access server corresponding to the organization.

[0071] The storage module of the access server stores the operation record data in the cache, which is intended to buffer the data when the amount of burst data is large.

[0072] When it is necessary to restore the storage order of the operation record data during query, the access server adds a serial number as a data prefix to each operation record when putting the operation record into the cache. The serial number is also stored in the blockchain network as part of the operation record data.

[0073] The storage module of the access server takes out the first operation record from the cache, selects a storable blockchain network from the blockchain network list according to the blockchain network selection strategy, and sends the operation record message to the corresponding node of the server in the network (by calling the smart contract deployed on the node).

[0074] As a preferred solution, the encoding method is to encode all possible fields, including but not limited to system, operation, and operation object, and the encoding of each type of field is represented in binary. The fields are spliced ​​together to generate the encoding to compress the data volume.

[0075] As a preferred solution, the blockchain network selection strategy can be a polling method, or it can be a priority order given by combining various factors such as storage space and performance indicators.

[0076] As a preferred solution, the corresponding nodes in the blockchain network receive operation records through smart contracts, and the operation records will be subsequently stored on each node in the network after consensus by the blockchain network.

[0077] As a preferred solution, a method for querying operation record data includes the following steps:

[0078] The external information system calls the query client to perform operation record query and transmits query parameters to the query client.

[0079] The query client processes the incoming query parameters. If the query parameters contain fields that were encoded during storage, the query request information for the query conditions corresponding to the encoded fields is the query index data generated for all matching fields. Each query condition is queried once, and the results of multiple queries are aggregated into the final query result. (For example, if the encoded fields include system, operation, and object, and the codes generated for these three fields are stored as a single field during storage, and the query parameters contain specific system and object information but not operations, the generated query conditions are the encoded data corresponding to all operation information for the object under the system. For details, please refer to the query examples in the embodiments.)

[0080] Because all encoded fields are stored as a whole, it is necessary to construct a query request for the entire field. The query condition is all query index data generated for these fields that can match these fields. The method for constructing the encoded field query request is as follows:

[0081] Assume that there are n fields stored in the encoding, and m query fields are used, denoted as t1, t2, ...t m , for any query field t i,i∈(1,2,…m), according to the value xi of the field passed in during the query, the bijective function f defined during storage is used to calculate f(x i )The value y obtained i For its corresponding code, m fields get a total of m codes. If m=n, that is, all fields in the code are queried, then these m codes are spliced ​​in the order they are stored to get the query value.

[0082] If there are nm fields besides the queried field, record them as t1', t2', ... t n-m ', for any non-query field t j ',j∈(1,2,…nm), let the field have k j Different values, according to the encoding scheme when storing, each value corresponds to a code, let the encoding set of the field be Sj', then the nm fields generate a total of nm encoding sets. Each field t of the m fields previously queried i Determines a coded value y based on the incoming query value xi i ,y i Can form a single value set {y i}, denoted as S i Now sort these n sets according to the order of the corresponding fields when they are spliced, and the re-sorted sets are recorded as S1, S2, ... S n , then do the Cartesian product S1×S2×…×S n , suppose there are z elements in the result set of Cartesian product, and any element is recorded as <y1,y2,…y n >, y1,y2,…y n A query code can be obtained by splicing, and a total of z query codes can be obtained.

[0083] The query client constructs query request information from all uncoded fields in the query conditions and z query codes for the coded fields to the access server. The access server selects the corresponding blockchain network from all blockchain lists based on the time period field. If there is no time period information, all blockchain networks are selected, and the blockchain network node information accessible to the access server is found based on the blockchain network.

[0084] The access server sends the query information and access credentials to each blockchain network node for querying. All uncoded fields in the query condition are combined with each of the z query codes for the coded fields, requiring z queries.

[0085] The blockchain network node returns the query result to the access server, and the access server merges the query results on each blockchain network and sorts them by timestamp.

[0086] As a preferred solution, optionally, when the query result needs to strictly restore the operation record order, for data with the same timestamp, the serial number added when entering the cache is used for sorting.

[0087] As a preferred solution, the access server returns the query result list to the query client, and the query client needs to process each piece of data and restore it to the original data format as follows:

[0088] Decryption: If part of the operation record information is encrypted before storage, the encrypted fields need to be decrypted accordingly.

[0089] Decoding: Decode the fields that are combined and coded in the operation record. Assume that there are n coded fields. Get the field code value y for the position and number of coded bits of any field t during coding. According to the inverse function f of the bijective function f defined during coding, -1 , calculate f -1 (y) obtains the value x, which is the value corresponding to field t. Use the same method to obtain the corresponding value for each coded field.

[0090] Return a list of field names and corresponding values ​​to the caller, or display it directly.

[0091] A blockchain network expansion method comprises the following steps:

[0092] Triggering blockchain expansion: When the monitoring module of the blockchain management client detects insufficient disk space, it sends an alert to the blockchain management server; or when the access server detects that the amount of cached operation record data exceeds a threshold, it sends an alert to the blockchain management server. This alert notifies the blockchain management server that a new blockchain network needs to be expanded. Optional alert information includes, but is not limited to, the time, alarm type, and alarm reason. If the alarm reason is insufficient disk space, it indicates that the current blockchain network is unsuitable for further data storage. After the new blockchain is expanded and storage is switched, its status in the blockchain network list needs to be changed to unavailable for storage.

[0093] Sending a deployment request: The blockchain management server sends a deployment request to the blockchain management client deployed thereon according to the server information of each organization in the configuration management module.

[0094] The blockchain management client determines whether the current server disk space and performance indicators can deploy a new blockchain based on the request information, and returns a response with the corresponding content.

[0095] The blockchain management server selects one or more nodes from the nodes that respond to the deployment and constructs the corresponding deployment information. If no server that meets the conditions is found, an alarm message is sent to the system administrator. The deployment information includes the organization information corresponding to the server and the identifier of the blockchain network node to be deployed.

[0096] The blockchain management server sends a deployment instruction to the blockchain management client, where the deployment instruction includes the deployment information.

[0097] The blockchain management client constructs a corresponding configuration file and deployment script based on the deployment information. The configuration file and deployment script can also be generated by the blockchain management server and issued along with the deployment instructions. The blockchain network node is then deployed, and the smart contract required for storage and query operation records is installed on the blockchain network node.

[0098] The blockchain management client registers the user with the blockchain network node on behalf of the access server as the blockchain network node manager and obtains access credentials. The access credentials are used for identity authentication of the access server when storing or querying data on the blockchain network node.

[0099] The blockchain management client returns the information that the blockchain network node is successfully deployed to the blockchain management server, and returns the access credentials and deployment information.

[0100] Upon receiving the successful deployment message, the blockchain management server updates the blockchain network list and related blockchain network node information, and sends the updated blockchain network list and the corresponding information and access credentials of the newly added blockchain network nodes accessible to the access server to the access server. If the alarm reason is insufficient disk, indicating that the blockchain network in the alarm information is not suitable for further data storage, the update operation includes changing its status in the blockchain network list to non-storable.

[0101] As a preferred solution, after receiving the update message, the access server updates the blockchain network list and saves the newly added blockchain network node information and access credentials.

[0102] Beneficial Effects: The present invention provides an easily scalable and traceable data operation recording system and method based on an easily scalable blockchain architecture. The architecture includes modules such as a sending client, a query client, an access server, a blockchain network, a blockchain management server, and a client, as well as interaction schemes between them. Submodules of each module can be used as embodiments or optional schemes. Expansion schemes based on this architecture include blockchain network management schemes, as well as expansion methods and processes. When the amount of real-time stored data increases, or blockchain storage efficiency decreases, or blockchain node disk space is insufficient, the blockchain network can be expanded automatically or notified to the administrator to meet data storage needs. Data encoding schemes and corresponding storage and query schemes can be used to reduce data volume.

[0103] Compared with the existing technology, its advantages are as follows:

[0104] 1. A blockchain-based, easily scalable, and traceable data operation recording platform is proposed. Because blockchain is decentralized, each peer node stores the same data, which is resistant to destruction and tampering. Key operation information in the system is recorded on the blockchain, which can be used to trace back problems when they occur.

[0105] 2. Existing blockchain operation record platforms either store the entire text of the operation log, which can easily fill up the hard drive, or only store summary content on the blockchain, while the log data is stored off-chain. This only verifies whether the log data has been tampered with, but if the log data is damaged, the content of previous operations cannot be determined. This solution proposes a data encoding scheme and a corresponding query scheme that can store the entire content of the operation log on-chain, providing tamper-proof and anti-destruction effects.

[0106] 3. Compared with the existing blockchain operation record platform, this paper proposes a set of easily scalable architecture solutions. When the amount of real-time stored data increases, or the blockchain storage efficiency decreases, or the blockchain node disk space is insufficient, the blockchain network can be expanded automatically or the administrator can be notified to meet the data storage needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0107] Figure 1 Schematic diagram of the overall architecture of the system of the present invention.

[0108] Figure 2 This is a structural diagram of the first embodiment of the sending client module.

[0109] Figure 3 This is a structural diagram of the second embodiment of the sending client module.

[0110] Figure 4 This is a structural diagram of the third embodiment of the sending client module.

[0111] Figure 5 This is a structural diagram of the query client module.

[0112] Figure 6 A schematic diagram of the structure of accessing the server module.

[0113] Figure 7 This is a structural diagram of the blockchain management server.

[0114] Figure 8 Expand the process sequence diagram for the blockchain network.

[0115] Figure 9 Schematic diagram of coding and splicing for each field.

[0116] Figure 10 Stores timing diagrams for operation records.

[0117] Figure 11 Query the timing diagram for the operation record. DETAILED DESCRIPTION

[0118] The present invention will be further described below with reference to specific embodiments.

[0119] like Figure 1 As shown in Figure 1, a scalable and traceable data operation record system includes an operation record platform, which consists of an access server, a blockchain network, a blockchain management server, and a blockchain management client. External information systems can send operation records to the access server through a sending client, or query the access server for a list of qualified operation records through a query client.

[0120] The access server receives the operation records sent by the sending client, selects a blockchain network from all blockchain networks according to the sending strategy, and sends the operation records to the blockchain network for storage; it can also accept the query request issued by the query client, and select one or more blockchain networks for query according to the query conditions, and merge and return the query operation record list in the order of storage.

[0121] The access server can manage which sending and query clients can access it according to the configured access rules.

[0122] Multiple access servers can be configured. When a blockchain is a consortium chain, it supports collaboration among multiple organizations. Each access server corresponds to one of these organizations. Each organization can customize access policies based on its needs and manage the list of accessible information systems. When a blockchain is a private chain, you can deploy only one access server, or multiple access servers depending on the usage scenario.

[0123] The blockchain management server and blockchain management client are used to manage the blockchain network, including installation, deployment, maintenance, deletion and other operations.

[0124] There can be multiple blockchain networks, including blockchain networks that can be stored continuously and blockchain networks that cannot be used for continued storage due to disk space fullness or other reasons. All blockchain networks can be used for query.

[0125] The blockchain management server allows administrators to manage blockchain networks, including deploying new and deleting them. It also configures rules for automatically deploying new networks under certain conditions. Deployment, deletion, and some configuration operations are delegated to the blockchain management client installed on the server. The blockchain management server maintains storable and queryable blockchain network information and synchronizes this information with all access servers. If an access server detects that it is receiving messages too quickly to store them, it issues an alert to the blockchain management server. Based on the configuration, the blockchain management server automatically deploys a new blockchain network or notifies the administrator to allocate storage capacity.

[0126] Nodes in a blockchain network accept requests to store operation records through smart contracts. Consensus nodes reach consensus and, once a certain amount of data has accumulated, the operation records are packaged into blocks and broadcast to all nodes for storage. Nodes in different blockchain networks can be deployed on the same server (which can be a physical machine or a cloud server).

[0127] The blockchain management client is deployed on a server. Its operations and maintenance module regularly checks performance data such as disk space. If insufficient disk space or other performance indicators are detected, it notifies the blockchain management server. The blockchain management server, based on its configuration, automatically or notifies the administrator to allocate a new server to deploy a new blockchain network. After deployment, the blockchain management client requests access credentials from the blockchain network node on that server and sends them to the blockchain management server. The blockchain management server then sends them to the access server, which uses them to access nodes in the blockchain network.

[0128] Example 1:

[0129] External information systems send operation records to the operation record platform through the sending client. The sending client encodes the operation records, which compresses the information and also provides a certain encryption effect compared to text data during network transmission and storage on the blockchain. To address the low storage efficiency of blockchains caused by the blockchain consensus algorithm, the access server can send operation record data to multiple blockchain networks simultaneously and monitor storage efficiency in real time using the built-in message cache. When the cache content is too large, an alarm is issued to the blockchain management server. The blockchain management server automatically deploys a new blockchain network or notifies the administrator based on the configuration to alleviate storage pressure.

[0130] Optionally, although the operation record records the timestamp, in order to restore the original order when querying, the operation record is added with a sequence tag when entering the access server cache.

[0131] To address the issue of operation records becoming ineffective over time, you can configure a policy on the blockchain management server to periodically generate a new blockchain network and modify the blockchain network list. This new blockchain network replaces the existing storable blockchain network as the new storable blockchain network, leaving the original blockchain network for query purposes only. If the physical machine hosting a blockchain network runs out of disk space, a similar policy can be employed: a new blockchain network can be created on a new physical machine and the existing storable blockchain network replaced.

[0132] External information systems send operation records to the operation record platform through the "sending client". There are three types of sending clients:

[0133] The first sending client is integrated as a plug-in by the information system, such as Figure 2 As shown, the data is transmitted to the first sending client by calling the interface provided by the first sending client plug-in. The operation record is encoded by the encoding module. Optionally, the encryption module encrypts part of the operation record information (the part that is not used as a query condition, or the part that can be used as a query condition but can be queried using ciphertext). The message sending module then sends it to the access server.

[0134] The second sending client does not invade the information system, such as Figure 3 As shown, the information system sends the operation records to the second sending client through the interface. The formatting module customized by the second sending client for the information system converts the operation records generated by the information system into data that can be encoded and then sends it to the encoding module. The subsequent process and modules are the same as those of the first sending client.

[0135] The third sending client does not invade the information system, such as Figure 4As shown, the information system writes the operation records into a log file or memory, and then a customized reading module reads the operation records from the log file or memory, performs certain information filtering, and then sends them to the formatting module for content conversion. The subsequent processes and modules are consistent with the second sending client.

[0136] like Figure 5 As shown, the external information system sends a message to the access server via the query client. The query client encodes a portion of the query keyword through the encoding module. The portion of the query keyword is the keyword encoded during storage. Optionally, when searching using ciphertext, the encoded keyword is encrypted. The query keyword is sent to the access server via the sending module. After the access server returns a message, the return message is obtained by the receiving module, optionally decrypted by the decryption module, and then decoded by the decoding module. The decoded operation record information is then returned.

[0137] like Figure 6 As shown, the access server includes an access control module, a storage module, a query module, and a blockchain interaction module.

[0138] The access control module is used to manage which information systems can access the access server. It optionally includes a list of accessible user identities and each user's permissions, such as storage permissions, query permissions, management permissions (including all permissions such as configuration and management), etc. Each user can have one or more permissions. Optionally, member permissions are achieved by adding the member's public key (also known as a certificate, signature certificate, or visa) to the corresponding permission list. The access control module includes a CA (Certificate Authority). Users apply for certificates from the CA, and the CA is responsible for issuing private keys to users.

[0139] Storage module: It includes four submodules: receiving module, policy module, sending module, and early warning module. The receiving submodule accepts storage requests from external information systems (which require storage permissions) and puts the messages into the cache. The sending module reads the first operation record data entered from the cache, selects a blockchain network from the list of all storable blockchain networks according to the sending strategy formulated by the policy module, reads the node information, and calls the blockchain interaction module based on the node information. The blockchain interaction module sends the operation record to the blockchain node for storage. The strategy in the policy module can be configured by the administrator. The strategy can be polled or selected according to the configured priority. The early warning module monitors the cache area. When it finds that the amount of data in it exceeds the threshold, it will issue an alarm to the blockchain management server.

[0140] Query Module: The query module consists of four submodules: the receiving module, the query module, the integration module, and the return module. The receiving module accepts query requests from external information systems (which require query permission). Based on the time period information in the query request, the query module selects a list of blockchain networks from all blockchain networks that contain data for that time period. This list contains information about blockchain nodes accessible to the server. This blockchain node information includes the IP address and port number for accessing the blockchain node. The query request and the blockchain node information in the list are sequentially sent to the blockchain interaction module, which then sends the query request to the blockchain node for querying. The query results are then passed to the integration module for integration. The integration module sorts the returned operation record list by timestamp. If the timestamps are the same and the data is from the same blockchain network, the order is followed by the original order. If the data is from different blockchain networks and has been tagged, the order is followed by the tag. The integration module passes the sorted operation record list to the return module for return to the external information system.

[0141] Blockchain Interaction Module: This module manages the identity credentials for blockchain node interactions, namely the private keys issued by the blockchain network CA. It is responsible for sending data to blockchain network nodes to submit transactions and querying the corresponding data transaction information list from blockchain network nodes based on query conditions.

[0142] This optional module is an SDK module provided by the operation recording platform to the application.

[0143] like Figure 7 As shown in the figure, the blockchain management server includes three sub-modules: deployment module, deletion module, and configuration management module.

[0144] Deployment Module: This module sends deployment instructions to the blockchain management client on the server. These instructions include the blockchain network's networking information and the organizational structure of each node. Administrators can deploy the module by calling its API. When the system is configured to allow automatic deployment, the deployment process can also be initiated by receiving alerts from the access server and blockchain management client. If automatic deployment is not configured, an alert will be sent to the administrator to request manual deployment. When generating blockchain network information, the deployment module combines server information in the configuration management module and performance data obtained from the blockchain management client to ensure that the servers belonging to the new blockchain network meet blockchain operation requirements. This information can also be generated by administrator configuration. Organizational structure information is also obtained from the configuration management module.

[0145] Configuration Management Module: This module is used by the administrator to set organizational structure information, that is, the organizational information of different alliance members. The administrator or alliance organization imports the server information (which can be a physical machine or a cloud server) used by each alliance member. When creating a blockchain network, the deployment module will generate blockchain network node information based on these two pieces of information (server information and organizational structure information). The configuration management module manages the currently storable blockchain node information of each alliance member, as well as the total blockchain node information, and records the time range corresponding to each blockchain node data. When the deployment module deploys a new blockchain network, it sends a message to the configuration management module to update the currently storable blockchain network information. The configuration management module will apply for access permission to the blockchain network on behalf of each organization through the configuration module of the blockchain management client and send it to the access server.

[0146] Deletion module: This module is used by administrators to delete invalid blockchain networks and delete invalid blockchain network nodes through the blockchain management client.

[0147] Example 2:

[0148] Taking the blockchain management technology solution as an example, set all blockchain network columns as shown in Table 1:

[0149] Table 1 is an example of a list of all blockchain networks

[0150]

[0151] Status 0 means it can be stored, 1 means it cannot be stored but can be read, and 2 means it is invalid.

[0152] By querying all lists with status 0, you can get a list of storable blockchains. This list is stored in the blockchain management server and synchronized to the access server.

[0153] Set the blockchain network information table properties, as shown in Table 2.

[0154] Table 2 is an example of blockchain network information table

[0155]

[0156] A blockchain node is uniquely identified by its blockchain node ID and blockchain network ID. The organization to which the node belongs is the organization information. Since the organization corresponds to the access server, the access server ID is also included. The IP is the IP address of the server where the node is deployed. The table also contains multiple management ports. Records in this table are generated by the blockchain management server when a new blockchain network is created. The corresponding deployment files or information are sent to the blockchain management client for deployment. Optionally, when the blockchain network is fabric, the deployment file is a shell script file and its corresponding YAML or JSON format file for storing configuration information.

[0157] Example 3:

[0158] like Figure 8 As shown, a blockchain network expansion method includes the following steps:

[0159] Step 1: The monitoring module of the blockchain management client detects insufficient disk space and sends an alarm to the blockchain management server.

[0160] Step 2: The blockchain management server sends a deployment request to the blockchain management client deployed on it based on the server information of each organization in the configuration management module.

[0161] Step 3: The blockchain management client confirms that the new blockchain node can be deployed and returns the request.

[0162] Step 4: The blockchain management server selects one or more nodes from the deployable blockchain nodes and constructs the corresponding deployment information.

[0163] Step 5: The blockchain management server sends deployment information to the blockchain management client.

[0164] Step 6: The blockchain management client constructs the corresponding configuration file based on the deployment information, deploys the blockchain network node, and installs the smart contract required to store operation records.

[0165] Step 7: The blockchain management client registers the user to the blockchain node on behalf of the access server as an administrator and obtains access credentials (such as the public key and certificate issued by the node CA) for authentication when accessing the server to the node.

[0166] Step 8: The blockchain management client returns the node deployment details to the blockchain management server and returns the access credentials.

[0167] Step 9: After receiving the successful deployment message, the blockchain management server updates the blockchain network information table. When all nodes are successfully deployed, the blockchain network list is updated.

[0168] Step 10: The blockchain management server sends the blockchain network information to the access server.

[0169] Step 11: The blockchain management server sends the node information and access credentials corresponding to each organization to the blockchain management server.

[0170] Step 12: Access the server to update the blockchain network and node information as well as access credentials.

[0171] Example 4:

[0172] Encode the fields whose values ​​can be exhaustively enumerated (such as system, operation, operation object, etc.) with the following encoding method: suppose a certain exhaustible field t has n different values, and the set of values ​​is defined as X. Now construct a binary encoding set Y of n numbers from 0 to n-1, and define a bijective function f from X to Y so that any value x in X can be obtained by f(x) to obtain a binary encoding y in the set Y. According to any encoding y in Y, the inverse function of f (denoted as f) can be used to obtain the binary encoding y in the set Y. -1 ), calculate f -1 (y) gets the value x. Let k be a value that satisfies 2 k If the smallest natural number ≥ n, then k bits can be used to store these n codes. Therefore, field t can use k bits to store its corresponding code. In actual use, field values ​​can also be reserved.

[0173] Assume that there are n fields whose values ​​can be exhaustively enumerated. Each field i is binary-encoded using the above method. The number of bits occupied by each field is ki. The codes of these fields can be generated by concatenating the fields. The number of bits occupied by the code is the sum of k1 to kn, that is, like Figure 9 The following shows an example of encoding and splicing of each field.

[0174] Take the operation record encoding scheme as an example:

[0175] For example, the operation record fields include (timestamp, system ID, user ID, operation behavior, operation object, and quantity). The timestamp and quantity are not encoded, and the user ID can be encoded according to the specific situation. The encoding scheme for other fields is shown in 3:

[0176] Table 3 is an example of the operation record coding scheme

[0177]

[0178]

[0179] As shown in the table, the system field requires 2 bits of encoding, the operation field requires 2 bits of encoding, and the operation object field requires 4 bits of encoding, totaling 8 bits, 1 byte. More bits can be allocated as reserved in actual operations.

[0180] For example, if user ID 1 changes the quantity of item 7 to 20 in system 1, the system, operation, and operation object fields are encoded as 01010110 in binary, or 5A in hexadecimal. The entire record is: (Time: 2022-12-21-09:00:01, User: 1, Encoding Field: 5A, Quantity: 20). In this example, only the system, operation, and operation object fields are encoded; other fields can be encoded as needed.

[0181] When the blockchain network uses fabric2.x, data can be stored in couchDB, and multiple index fields can be used for querying.

[0182] 5A is recorded as a query index for this operation in the blockchain. Similarly, all operations on all items in all other systems are encoded accordingly, and the hexadecimal representation corresponding to the code is used as the query index.

[0183] Example 5:

[0184] Example of an encoded query scheme:

[0185] Example 1: When the query condition is the modification operation on item 7 in system 1, this record can be found through index 5A.

[0186] Example 2: When the query condition is the modification operation in system 1, all coded items will be queried, and the query indexes are 51, 52, 53...5A in sequence, and all query results are merged into the final result.

[0187] Example 3: When the query condition is all operations on item 3 in system 1, the binary value corresponding to the add operation is 01000010, the modify operation is 01010010, and the delete operation is 01100010, with indexes of 42, 52, and 62 respectively. Combine all query results into the final result.

[0188] Example 6:

[0189] Taking the operation record access scheme as an example, it includes: a number label method, an operation record storage method and an operation record query method.

[0190] The numbering labeling method includes the following steps:

[0191] Consecutive operation records may be stored across multiple blockchain networks, and the same timestamp may correspond to multiple records. Timestamps alone may not necessarily restore the order in which the operation records were generated. Optionally, when restoring the storage order of operation record data is required during querying, a sequence number can be added to each operation record before or during cache entry to identify the order in which the operation records were generated.

[0192] The sequence number is represented by a 2-byte short int, starting at 0 and re-starting at 0 when the timestamp exceeds 65535. Ambiguity is eliminated when the number of data within the same timestamp is significantly less than 65536. Assuming the maximum data size is 10,000, if only one of 65535 and 0 exists, the numbers are sorted in that order. If both exist, numbers between 0 and 10,000 are sorted after 65535.

[0193] like Figure 10 As shown, the operation record storage method includes the following steps:

[0194] Step 1: The information system transmits the data to the sending client to store the operation record. This step can also be that the sending client reads the operation record in the information system.

[0195] Step 2: The sending client processes the operation record data, including filtering (retaining only key operations), formatting (converting to a processable format), encoding (referring to the data encoding scheme), and encryption (optional operation).

[0196] Step 3: The sending client sends the processed operation record to the access server corresponding to the organization.

[0197] Step 4: Access the server to store the data in the cache. Optionally, when it is necessary to restore the storage order of the operation record data during query, a serial number is added to each operation record when it is placed in the cache. The serial number will also be stored in the blockchain network.

[0198] Step 5: The access server retrieves the first operation record from the cache, selects a blockchain network from the list of storable blockchain networks based on the blockchain network selection strategy (blockchain network 1 in the example), and sends the operation record message to the corresponding node of the access server in the blockchain network (by calling the smart contract deployed on the node).

[0199] Step 6: The corresponding node in the blockchain network (blockchain network 1 in the example) receives the operation record through the smart contract. The operation record will be stored on each node in the blockchain network after consensus is reached by the blockchain network.

[0200] Step 7: When there are multiple blockchain networks, each blockchain network can receive operation record messages simultaneously as in steps 3-6 to improve storage efficiency.

[0201] like Figure 11 As shown, the operation record query method includes the following steps:

[0202] Step 1: The information system calls the query client to query the operation record and passes the query parameters to the plug-in.

[0203] Step 2: The query client processes the query parameters and encodes the corresponding query parameters for the encoded fields, as shown in the example in Example 5.

[0204] Step 3: The query client sends a query request to the access server.

[0205] Step 4: Access the server and select the corresponding blockchain network from the list of all blockchains based on the time period field (in this example, blockchain networks 1 and 2 are matched) and find an accessible node.

[0206] Step 5: Send the query information and access credentials to the blockchain network 1 node for query.

[0207] Step 6: Send the query information and access credentials to the blockchain network 2 node for query.

[0208] Step 7: Merge the query results on each blockchain network and sort them by timestamp. Optionally, when the operation record order needs to be strictly restored, sequence tags are used for sorting, as shown in Example 6.

[0209] This invention establishes a blockchain-based, easily scalable, and traceable data operation recording platform. Because blockchain is decentralized, each peer node stores the same data, making it highly resistant to destruction and tampering. Key operational information in the system is recorded on the blockchain, allowing for backtracking when problems arise. Furthermore, to address the blockchain's low storage efficiency and the large disk space occupied after long runs, a data encoding scheme and corresponding query scheme are proposed. These schemes can store the entire content of operation records on-chain, providing tamper-proof and anti-destruction capabilities. Furthermore, a scalable architecture solution is proposed. When the amount of real-time stored data increases, blockchain storage efficiency decreases, or blockchain node disk space becomes insufficient, the blockchain network can be expanded automatically or by notifying the administrator to meet data storage needs.

[0210] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for querying operation record data, characterized in that: The steps include: Step 1: The external information system calls the query client to query the operation record and passes the query parameters to the query client; Step 2: The query client processes the incoming query parameters. If any query parameters contain fields that were encoded during storage, the query client constructs a corresponding query request message for the encoded fields. The steps for constructing the encoded field query request message are as follows: Assume there are n fields to be encoded and stored, of which m are query fields, denoted as t1, t2, ..., tm. For any query field ti, i∈(1,2, ..., m), according to the value xi passed in during the query, the bijective function f defined during storage is used to calculate the value yi obtained by f(xi) as its corresponding code. A total of m codes are obtained for m fields. If m=n, that is, all fields in the code are queried, then these m codes are concatenated in the order in which they were stored to obtain the query request information; If there are nm fields other than the queried field, denoted as t1', t2', ... tn-m', for any non-queried field tj', j ∈ (1, 2, ..., nm), let the field have kj different values. According to the encoding scheme during storage, each value corresponds to a code. Let the code set of the field be Sj'. Then, a total of nm code sets are generated for the nm fields. Each field ti of the queried m fields determines a coded value yi according to the incoming query value xi, and yi forms a single-value set {yi}, denoted as Si; Arrange n code sets in the order of corresponding fields when they are concatenated. The re-arranged sets are denoted as S1, S2, ... Sn. Cartesian product S1 × S2 × ... × Sn is performed. Suppose there are z elements in the result set of Cartesian product. Any element is denoted as<y1,y2,…,yn> , concatenate y1, y2, ..., yn to obtain a query code, and obtain z query codes in total. The z query code fields and other unencoded query fields together constitute the query request information; Step 3: The query client sends a query request to the access server. The access server selects the corresponding blockchain network from the list of all blockchains based on the time period field. If there is no time period information, all blockchain networks are selected and the blockchain network node information accessible to the access server is found based on the blockchain network. Step 4: The access server sends the query request information and access credentials to each blockchain network node for query; all uncoded fields in the query condition are combined with each of the z query codes for the coded fields, requiring z queries; Step 5: The blockchain network node returns the query results to the access server. The access server merges the query results on each blockchain network and sorts them by timestamp.

2. The operation record data query method according to claim 1, characterized in that: Also includes: When the query results need to restore the order of operation records, for data with the same timestamp, they are sorted using the serial number added when entering the cache.

3. The operation record data query method according to claim 1, characterized in that: Also includes: The access server returns the query result list to the query client. The query client needs to process each piece of data and restore it to the original data format, including the following steps: If part of the operation record information is encrypted before storage, the encrypted fields are decrypted accordingly; Decode the fields that are combined and coded in the operation record; suppose there are n coded fields, and obtain the field code value y for the position and number of coded bits of any field t during coding, and use the inverse function of the bijective function f defined during coding ,calculate Get the value x, then x is the value corresponding to field t; use the same method to obtain the corresponding value for each coded field.

Images