A system and method for checking non-original data features based on blockchain

Abstract

The present invention relates to the technical field of data compliance verification, and specifically discloses a system and method for non-original data feature verification based on blockchain, including a smart contract with a two-dimensional splitting rule arranged at the bottom layer of a locally deployed data feature verification terminal. The verification method is to perform two-dimensional rule splitting on user input data and locally stored original data in combination with the smart contract, and compare the hash values ​​converted after the splitting, which solves the problem of traditional data feature verification methods that due to the need to release data from the warehouse, both the original data to be verified and the original data to be compared need to be stored in a fixed system platform, which is prone to data privacy leakage.

Description

Technical Field

[0001] The present application relates to the technical field of data compliance verification, and specifically discloses a system and method for non-original data feature verification based on blockchain. Background Art

[0002] Currently, machine verification of data content similarity is mainly achieved in two ways:

[0003] The first is to use traditional software to perform feature verification, that is, to develop software applications for feature verification data, then store all the original data that need to be compared in the software, use computer programs to compare them one by one, and return the verification and comparison results;

[0004] Another method is to perform data duplication check by performing hash feature comparison calculations on the sharded areas according to the amount of data in the data set.

[0005] The main disadvantages are as follows:

[0006] In traditional data feature verification, all data needs to be stored in the warehouse, and both the original data being verified and the original data being compared need to be stored in a fixed system platform. Compared with directly verifying data on a locally deployed data feature verification terminal, the risk of data security and privacy leakage is very high.

[0007] Comparing the feature hashes of shards based on the amount of data in a data set is basically not practical for locating and analyzing malicious data changes and tampering, or for repeated verification of modified single data entries. This is because once the data processor inserts a modification unit into each piece of data, this comparison method becomes completely ineffective.

[0008] Therefore, in view of this, the inventor provides a system and method for non-original data feature verification based on blockchain to solve the above problems. Summary of the invention

[0009] The purpose of the present invention is to solve the problem that in traditional data feature verification methods, due to the need for data to be released from the warehouse, both the original data to be verified and the original data to be compared need to be stored in a fixed system platform, which is prone to cause data privacy leakage.

[0010] In order to achieve the above-mentioned purpose, the basic scheme of the present invention provides a non-original data feature verification system based on blockchain, including a smart contract with two-dimensional splitting rules arranged at the bottom layer of a locally deployed data feature verification terminal, and the two-dimensional splitting rules include single data splitting according to data fields and data particle splitting of single data, which are executed sequentially.

[0011] Furthermore, the data subsets split according to the dual-dimensional splitting rule need to be numbered according to the splitting order, and the data that do not belong to the same subset are renumbered in sequence.

[0012] The basic scheme of the present invention also provides a verification method of a non-original data feature verification system based on blockchain, comprising the following steps:

[0013] Step S001: The locally deployed data feature inspection terminal receives user input data to generate a data feature inspection instruction, and the system calls the smart contract interface to send a data content splitting instruction;

[0014] Step S002: The data content splitting instruction controls the feature checking system to split the user input data according to the two-dimensional rules, and to number the split two-dimensional subsets according to the rules and generate a hash subset bound to the data splitting number;

[0015] Step S003: The generated feature hash subset is checked for the same hash as the feature hash pool stored locally in the terminal, and can be compared with the historical hash uploaded by the terminal to the feature hash fingerprint library, and the produced feature hash set is written into the blockchain as the compared feature hash fingerprint library.

[0016] Furthermore, it also includes data duplication checking. During data duplication checking, the checked data set needs to be split according to the two-dimensional splitting rules, and hash values ​​need to be generated in segments.

[0017] Furthermore, the data duplication check includes internal duplication check, external duplication check and location data tampering location check.

[0018] Furthermore, the internal duplication check includes data duplication check, sensitive word check and data null value check. The sensitive word check judgment method is to check the built-in sensitive word hash calculation results of the feature hash subset and the feature hash pool, and the data content null value check judgment method is to check the length of the feature hash subset and the feature hash pool.

[0019] Furthermore, the process of external repeated checking is: checking the partition hash set of the data field to be checked with the hash set in the feature hash pool, retrieving the hash set containing the same hash and confirming similar data sources.

[0020] Furthermore, the process of locating the data tampering position is: comparing the subset hash in the data field partition hash set to be checked with the hash set obtained by the same historical algorithm, numbering and marking different abnormal hash subsets, and finding the corresponding numbered subset using the two-dimensional splitting rule.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. The present invention stores the original data in a terminal where data feature verification is locally deployed. The user can complete the verification steps directly on the terminal. Without the data being stored out of the warehouse, only the hash algorithm is used to realize efficient and accurate feature verification of large-scale structured data. This solves the problem that the traditional data feature verification method needs to store the data out of the warehouse, and both the original data to be verified and the original data to be compared need to be stored in a fixed system platform, which easily leads to data privacy leakage.

[0023] 2. The present invention performs dual-dimensional splitting of user data and original data through dual-dimensional splitting rules, and can perform highly accurate inspection on each piece of data.

[0024] 3. After checking the data input by the user, the present invention will write the feature hash set generated by the new data into the blockchain as the compared feature hash fingerprint library to enrich the local feature hash fingerprint library.

[0025] 4. After the present invention splits the data using the dual-dimensional splitting rule, it will label the data and bind it with the hash subset, which can facilitate rapid positioning of the data by numbering during subsequent data inspection.

[0026] Compared with the prior art, the present invention can realize efficient and accurate feature verification of large-scale structured data by only using a hash algorithm without releasing the data from the warehouse, and is the first to provide a two-dimensional splitting method for data verification, solving the problem that traditional data feature verification methods require data to be released from the warehouse, and both the verified original data and the compared original data need to be stored in a fixed system platform, which easily leads to data privacy leakage. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 The schematic diagram of the architecture of a system and method for checking non-original data features based on blockchain proposed in an embodiment of the present application is shown;

[0029] Figure 2 A flowchart of a system and method for checking non-original data features based on blockchain proposed in an embodiment of the present application is shown;

[0030] Figure 3 A flowchart of a system and method for checking non-original data features based on blockchain proposed in an embodiment of the present application is shown;

[0031] Figure 4 A flowchart of a system and method for checking non-original data features based on blockchain proposed in an embodiment of the present application is shown;

[0032] Figure 5 A flowchart of a system and method for checking non-original data features based on blockchain proposed in an embodiment of the present application is shown;

[0033] Figure 6 An identification diagram of a system and method for checking non-original data features based on blockchain proposed in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] The following is a further detailed description through specific implementation methods:

[0036] A non-original data feature verification system based on blockchain, such as Figure 1 and Figure 2 As shown, the system and method are applied to each enterprise's local deployment of data feature inspection terminals. The underlying blockchain in the terminal deploys a smart contract with a two-dimensional rule splitting of data content. The data content is split by two-dimensional rules, and then the hash value of the split data is written into the blockchain. When it is used to input inspection data in the terminal, after the user inputs the data, the input data will be split by two-dimensional rules and a hash subset will be generated. The data is compared by comparing the feature hash of the original data with the newly input data, and the data feature situation is confirmed;

[0037] A verification method of a non-original data feature verification system based on blockchain. The following are the specific steps of the real-time comparison verification method:

[0038] Step S001: The user inputs data, and the feature inspection system front-end module receives the user input to generate a data feature inspection instruction, and the system calls the smart contract interface to send a data content splitting instruction;

[0039] Step S002: The data content splitting instruction controls the feature inspection system to split the data input by the user according to the two-dimensional rules, and to number the split two-dimensional data subsets according to the rules, and to generate a hash subset, which is bound to the data splitting number;

[0040] Step S003: The generated feature hash set is verified to be the same as the feature hash pool stored locally in the terminal to complete the comparison data verification, and the generated feature hash set is written into the blockchain as the compared data hash set.

[0041] The two-dimensional data splitting rule is as follows: splitting of dimension one and dimension two in sequence.

[0042] Dimension 1: Split data content according to data fields:

[0043] First, split the single data content, and you can get the following single data set:

[0044] The first data field content [A1, B1, C1, D1, E1, F1...]

[0045] The second data field content [A2, B2, C2, D2, E2, F2...]

[0046] The content of the data field of the Nth article [An, Bn, Cn, Dn, En, Fn...]

[0047] Then split and combine all the data under the same field type to obtain the following field data set:

[0048] A[A1, A2, A3, ..., An]

[0049] B[B1, B2, B3, ..., Bn]

[0050] C[C1, C2, C3, ..., Cn] ...

[0051] Dimension 2: Split data into segments based on the number of data items

[0052] The field data set that has been split according to the rules of dimension one will be segmented according to the number of original data items. The number of data items in the segment starts from n and ends at 1. When the number of data items in the data segment is n, the data level is only split once. When the number of data items in the data segment is 1, the data level is split n times. The result of the n-times split is the smallest granularity that is consistent with the granularity of the original data. This method can achieve 100% accurate repeat inspection.

[0053] After the split is completed, the field classification data set is formed. The data subsets under different classification conditions are as follows:

[0054] Level 1:

[0055] A[A1, A2, A3, ..., An]

[0056] B[B1, B2, B3, ..., Bn]

[0057] C[C1, C2, C3, ..., Cn] ...

[0058] Level 2:

[0059] When n is an odd number

[0060] A[A1, A2, ..., A(n-1) / 2], [A(n-1) / 2, ..., A(n-1)], [An]

[0061] B[B1, B2, ..., B(n-1) / 2], [B(n-1) / 2, ..., B(n-1)], [An]

[0062] C[C1, C2, ..., C(n-1) / 2], [C(n-1) / 2, ..., C(n-1)], [An] ...

[0063] When n is an even number

[0064] A[A1, A2, ..., A(n) / 2], [A(n) / 2, ..., An]

[0065] B[B1, B2, ..., B(n) / 2], [B(n) / 2, ..., Bn]

[0066] C[C1, C2, ..., C(n) / 2], [C(n) / 2, ..., Cn] ...

[0067] Level n:

[0068] A[A1],[A2],[A3],...,[An]

[0069] B[B1],[B2],[B3],...,[Bn]

[0070] C[C1],[C2],[C3],...,[Cn] ...

[0071] The data in [] is a subset. A complete data set contains all subsets. The split data subsets are numbered according to the above rules.

[0072] When the data is repeatedly checked, it must be ensured that the segmented hash of the data set being checked, the number of data items in the segment area is consistent with the number of data items to be compared, and the segmented hash in the hash pool is consistent with the number of data items in the corresponding segment area. When the control is found by comparing the hash values, the subset number with control is specially marked, and the total number of hashes in the subset will be filled with the corresponding null value after the mark instead of the value K, and the number of K is the number of null values;

[0073] According to the above rules, the following comparative data checks can be performed: internal duplicate checks, external duplicate checks, and location checks for data tampering. Since the calculation process of the hash function is one-way and irreversible, and the same data generates a fixed hash value, and the two-dimensional splitting rules are paired with the hash value, the data can be accurately located and compared. Internal duplicate checks refer to: data feature checks within the original data, and external duplicate checks refer to: data set feature checks corresponding to the feature hash fingerprint library where all historical hash sets uploaded by the terminal are located;

[0074] like Figure 3 As shown in the figure, the internal duplication check includes data duplication check, sensitive word check and data content null value check. The sensitive word check process is: compare the data feature hash value obtained after the input data is split with the original data hash value. If they are consistent, it means that the original data contains sensitive words. The null value check process is: because null data has no hash value, the number of hash entries calculated by the data set is compared with the number of original data entries. If the number of hash entries is less than the number of original data entries, it can be identified as null data.

[0075] like Figure 5 As shown, internal data duplication can be checked by the following rules:

[0076] All subset hashes in the data set to be checked will be compared with other subset hashes. When segment hashes under the same field are repeated, the numbers corresponding to the repeated hashes of different partitions (such as A1, A2) of the same field (such as field A) are marked with φ(A1, A2), and the repeated hashes of the same segment (such as A1, B1) of other fields (such as fields A and B) are marked with λ(B1, A1), as follows Figure 6 As shown;

[0077] The first case: When φ(A1, A2) and φ(B1, B2) exist at the same time, it is judged that there is data duplication, that is, all the data in the first subset and the data in the second subset are the same data, otherwise it is judged that there is no data duplication.

[0078] The second case: when λ(C1, D1) exists, it is determined that the two data are repeated, that is, the first data of the original data is repeated data.

[0079] Here, A1, B1, ... are data subsets and are different from A1 (the minimum data factor of the original data set) in the above data classification.

[0080] like Figure 3 As shown, the process of external repeated verification is as follows: the data field partition hash set to be checked (named as data set a) uses the terminal interface to call the hash set in the feature hash pool for verification. When the subsets within the data set find the same hash subset in the hash pool, the same subsets in data set a are marked, and the hash set (named as data set b) where the same hash is located is retrieved to confirm similar data sources.

[0081] The similarity calculation method for external repeated checks is:

[0082] Assume that the number of labeled data subsets of data set a is X, and the total number of data subsets of a is Y. Then the proportion of repeated data in data set a is (X / Y)×100%

[0083] The process of locating the data tampering position is as follows: compare the subset hash in data set a with the hash set obtained by the same historical algorithm. If the subset hash is different, the abnormal subset number is marked as (aAn). Assuming that the A1 field of data set a is the first identified data set, it is marked as (aA1).

[0084] Assume that the field data set is split according to the two-dimensional splitting rule and the secondary split distribution is as follows:

[0085] A[A1, A2, ..., A(n) / 2], [A(n) / 2, ..., An]

[0086] B[B1, B2, ..., B(n) / 2], [B(n) / 2, ..., Bn]

[0087] C[C1, C2, ..., C(n) / 2], [C(n) / 2, ..., Cn]

[0088] [A1, A2, ..., A(n) / 2] is subset aA1, and [A(n) / 2, ..., An] is subset aA2. When subset aA1 is marked, it means that the A field data from the first to the n / 2th item of the original data have been tampered with. This method can directly and accurately locate the tampered fields and the corresponding number of items.

[0089] Compared with the prior art, the present invention can realize efficient and accurate feature verification of large-scale structured data by only using a hash algorithm without releasing the data from the warehouse, and is the first to provide a two-dimensional splitting method for data verification, solving the problem that traditional data feature verification methods require data to be released from the warehouse, and both the verified original data and the compared original data need to be stored in a fixed system platform, which easily leads to data privacy leakage.

[0090] The present invention stores the original data in a terminal where data feature verification is locally deployed. The user can complete the verification steps directly on the terminal. Without the data being stored out of the warehouse, only the hash algorithm is used to realize efficient and accurate feature verification of large-scale structured data. This solves the problem that the traditional data feature verification method needs to store the data out of the warehouse, and both the verified original data and the compared original data need to be stored in a fixed system platform, which easily leads to data privacy leakage.

[0091] Although the above methods are illustrated and described as a series of actions for simplicity of explanation, it should be understood and appreciated that these methods are not limited by the order of the actions, because according to one or more embodiments, some actions may occur in different orders and / or concurrently with other actions from the illustrations and descriptions herein or not illustrated and described herein but understandable to those skilled in the art. It will be further appreciated by those skilled in the art that the various illustrative logic blocks, modules, circuits, and algorithm steps described in conjunction with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of the two. To clearly explain this interchangeability of hardware and software, various illustrative components, frames, modules, circuits, and steps are generally described in the form of their functionality above. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. The technical staff can implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention. The various illustrative logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in an alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration. The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read and write information from / to the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside in a user terminal as discrete components. In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored or transmitted as one or more instructions or codes on a computer-readable medium.Computer-readable medium includes both computer storage medium and communication medium, and it includes any medium that facilitates computer program to transfer from one place to another place. Storage medium can be any available medium that can be accessed by computer. As an example and not limitation, such computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage or other magnetic storage device, or can be used to carry or store the desired program code of instruction or data structure form and any other medium that can be accessed by computer. Any connection is also properly referred to as computer-readable medium. For example, if software is transmitted from website, server or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology such as infrared, radio and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are just included in the definition of medium. Disk and disc as used herein include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, wherein disk often reproduces data magnetically, while disc reproduces data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

Claims

1. A verification method for a non-original data feature verification system based on blockchain, characterized in that: It includes a smart contract with a dual-dimensional splitting rule set at the bottom layer of a locally deployed data feature inspection terminal, and the dual-dimensional splitting rule includes a single data splitting according to data fields and a data granular splitting of the single data, which are executed in sequence; The data subsets split according to the two-dimensional splitting rules need to be numbered in the order of splitting, and the data that do not belong to the same subset should be renumbered in sequence; The inspection method includes the following steps: Step S001: The locally deployed data feature inspection terminal receives user input data to generate a data feature inspection instruction, and the system calls the smart contract interface to send a data content splitting instruction; Step S002: The data content splitting instruction controls the feature checking system to split the user input data according to the two-dimensional rules, and to number the split two-dimensional subsets according to the rules and generate a hash subset bound to the data splitting number; Step S003: The generated feature hash subset is checked for the same hash as the feature hash pool stored locally in the terminal, and is compared with the historical hash uploaded by the terminal to the feature hash fingerprint library, and the produced feature hash set is written into the blockchain as the compared feature hash fingerprint library.

2. The verification method of the non-original data feature verification system based on blockchain according to claim 1 is characterized in that: It also includes data duplication checking, which includes internal duplication checking, external duplication checking and data tampering location checking.

3. The verification method of the non-original data feature verification system based on blockchain according to claim 2 is characterized in that: Internal duplication checks include data duplication checks, sensitive word checks, and data null value checks. The sensitive word check judgment method is to check the built-in sensitive word hash calculation results of the feature hash subset and the feature hash pool. The data content null value check judgment method is to check the length of the feature hash subset and the feature hash pool.

4. The verification method of the non-original data feature verification system based on blockchain according to claim 2 is characterized in that: The process of external repeated verification is as follows: the partitioned hash set of the data field to be verified is checked with the hash set in the feature hash pool, the hash set containing the same hash is retrieved and similar data sources are confirmed.

5. The verification method of the non-original data feature verification system based on blockchain according to claim 2 is characterized in that: The process of locating the data tampering location is as follows: compare the subset hash in the data field partition hash set to be checked with the hash set obtained by the same historical algorithm, number and mark the different abnormal hash subsets, and use the two-dimensional splitting rule to find the corresponding numbered subset.

Images