A data collection system based on blockchain

Abstract

The present invention discloses a data collection system based on blockchain, which relates to the technical field of node data collection, and solves the problems of the original data collection method, which is too slow in data collection, has great difficulty in searching, and consumes too much computing power resources. The present invention determines the target node through the determined target data, and then determines the same type of nodes based on the target node, and confirms the relevant processing processes of the synchronization node and the target node in turn, and determines and reorganizes the relevant data based on the processing time, determines the source data, and then processes them one by one upward, so that the source data of the corresponding target data can be quickly traced and collected. This source data tracing collection and processing method is faster than the original collection and search method, and the accuracy can also be guaranteed, which is convenient for relevant collection of large-capacity data existing in large-capacity space, based on the processing process and processing characteristics of different data of its nodes.

Description

Technical Field

[0001] The present invention relates to the technical field of node data collection, and in particular to a data collection system based on blockchain. Background Art

[0002] Blockchain is a chain data structure that combines data blocks in a sequential manner in chronological order. During the relevant circulation process, the data within the blockchain is transmitted between relevant nodes.

[0003] The application with publication number CN113055180B belongs to the field of data acquisition technology, and in particular, it relates to a data acquisition system based on blockchain. It includes sensors, Beidou precise timing module, blockchain DTU, data center, blockchain data verification module and data distribution module; the sensor is connected to the blockchain DTU, and the blockchain DTU is respectively connected to the Beidou precise timing module and the data center, and the data center is connected to the blockchain data verification module and the data distribution module. The uniqueness, legality and time accuracy of the data collected by the front-end data acquisition device in the subsequent entire system are realized, so that the data collected by the front-end data acquisition system is more in line with the use security of conventional industries or industries with higher confidentiality requirements.

[0004] During the collection process of the relevant transmission data within the blockchain, the relevant operators determine the target data, and then the data is collected in each relevant node of the blockchain. However, this collection method is relatively slow because a large amount of data is recorded in the corresponding relevant nodes. The search-based collection method is difficult to search and cannot achieve better collection results. A better data collection method has not been set up to quickly collect and display the required data. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a blockchain-based data collection system, which solves the problems of the original data collection method, such as slow data collection, high search difficulty, and excessive consumption of computing resources.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: A data collection system based on blockchain, comprising:

[0007] The target data selection end determines the target data from the node associated with the blockchain based on the verification requirements or other requirements of the relevant operator, and transmits the determined target data to the associated node locking end, wherein the target data is determined by the operator himself;

[0008] The associated node locking end, based on the target data determined at the current moment, identifies whether the same target data exists at other nodes, and determines the same nodes as the target node, and transmits the confirmed same nodes and target nodes to the source data traceability management end. The specific method is as follows:

[0009] Based on the target data determined at the current moment, determine the target node related to the target data;

[0010] Based on the target node and the target data, identify the previous set of output nodes of the target data, and identify whether the output node sends relevant data to other nodes. If so, mark the other nodes as the same type of nodes. If not, no marking is performed.

[0011] The source data traceability management terminal determines the same type of related data based on the determined target node and similar nodes, and traces the source data based on the processing process of the data, and transmits the traced main source data to the verification terminal. The specific method is as follows:

[0012] For the situation where there are similar nodes: based on the determined target node and similar nodes, determine the target data selected by the target node, and then identify the relevant processing rates and the relevant data processed by other similar nodes;

[0013] Mark the current time as T0, determine the real-time processing rate V of the target data corresponding to the target node, and the completed processing volume CL of the corresponding target data, determine the time length of the target node to process the target data based on the completed processing volume CL and the real-time processing rate V, and then determine the time point T1 when the target node receives the initial source data of the target data based on the current time T0 and the time length;

[0014] According to the relevant processing rate and the processed relevant data of other similar nodes, the processing volume of the processed relevant data of the corresponding similar node is determined, and the time length of the corresponding similar node processing the corresponding relevant data is determined in combination with the relevant processing rate, and then based on the current time T0 and the time length, the time point T0 at which the corresponding similar node receives the initial source data of the relevant data is determined. i , where i represents different nodes of the same type;

[0015] Based on different time points T1 and T i , sort the corresponding original source data in order of time to obtain complete original source data;

[0016] Determine the relevant nodes of the initial source data, identify whether the source node of the initial source data sends relevant data to other nodes synchronously, if yes, mark the relevant node as the target node, mark other nodes as nodes of the same type, and confirm the data according to the processing process of the initial source data of the corresponding target node and the processing process of other data of the same type of node, and sort the confirmed data in chronological order to obtain the source data, and confirm them in turn, until the relevant node of the determined source data does not send data to other nodes synchronously, and mark the confirmed source data at this moment as the main source data of the target data;

[0017] In the case where there is no similar node: determine the original source data of the target data of the target node;

[0018] The relevant nodes of the initial source data identify whether the source nodes of the initial source data send relevant data to other nodes synchronously. If yes, the relevant nodes are marked as target nodes, and other nodes are marked as nodes of the same type. According to the processing process of the initial source data of the corresponding target node and the processing process of other data of the same type of nodes, data confirmation is performed, and the confirmed data is sorted in chronological order to obtain the source data, and confirmation is performed upward in sequence until the relevant nodes of the determined source data do not send data to other nodes synchronously, and the confirmed source data at this moment is marked as the main source data of the target data;

[0019] The verification terminal searches for the data records of the main source data and the corresponding nodes in the cloud database, identifies whether the main source data is accurate, and performs capacity verification. Based on the verification results, it is determined whether the data cleaning terminal needs to perform relevant cleaning on such main source data. The specific methods are as follows:

[0020] Based on the determined capacity Rz of the main source data, and based on the relevant nodes corresponding to the main source data, the data records of the relevant nodes about the main source data are determined, and from the data records, the original capacity Rs of the data is determined;

[0021] Identify whether Rz and Rs satisfy: (Rz-Rs)>Y1, where Y1 is a preset value. If so, the data cleaning end is executed to clean the main source data. If not, the main source data is directly displayed.

[0022] The data cleaning end cleans the confirmed main source data, identifies the relevant capacity of irrelevant data based on the relevant capacity difference of the main source data, and analyzes and processes the main source data to identify irrelevant data and remove them. The specific method is:

[0023] Based on the data capacity Rz of the main source data and the original capacity Rs, the capacity difference Cz=Rz-Rs is determined;

[0024] Then, the format of the main source data is converted, and its conversion rate is recorded in real time. The recorded conversion rate is subjected to data clustering processing. Based on the clustering results of the data, the conversion rate with abnormal values ​​is determined: a group of parameters with the most conversion rate occurrences is calibrated as standard parameters, and this standard parameter is used as the cluster center point. A group of floating intervals about this standard parameter is constructed, and the floating change value of the floating interval is a preset value. The conversion rate that does not belong to this floating interval is calibrated as an abnormal conversion rate;

[0025] Determine the relevant data corresponding to the abnormal conversion rate, and record the data capacity of the corresponding relevant data. If the data capacity = Cz, then directly remove such relevant data from the main source data. If the data capacity ≠ Cz, then generate an abnormal acquisition capacity signal and display it.

[0026] The present invention provides a data collection system based on blockchain. Compared with the prior art, it has the following beneficial effects:

[0027] The present invention determines the target node through the determined target data, and then determines the same type of node based on the target node, and confirms the relevant processing process of the synchronization node and the target node in turn, and determines and reorganizes the relevant data based on the processing time, determines the source data, and then processes it one by one, so that the source data of the corresponding target data can be quickly traced and collected. This source data tracing and collection processing method is faster than the original collection and search method, and the accuracy can also be guaranteed. It is convenient to carry out relevant collection in large-capacity data in large-capacity space, and the relevant data is traced based on the processing process and processing characteristics of different data of its nodes, and then the data is reorganized based on the different time points to determine the complete data;

[0028] For relevant source data with a large amount of irrelevant data, the format conversion method is used to determine the relevant rate, and then the abnormal rate is determined based on the determined relevant rate. Then, based on the determined abnormal rate, the irrelevant data is locked and eliminated. By using data clustering analysis and processing, such abnormal changes can be quickly and effectively found, and timely processed to lock the abnormal conversion rate, and quickly lock and eliminate irrelevant data to ensure the overall data quality of the main source data. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a schematic diagram of the principle framework of the present invention;

[0030] Figure 2 Schematic diagram of data transmission of nodes related to the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] See also Figure 1 , the present application provides a data collection system based on blockchain, including a target data selection end, an associated node locking end, a source data traceability management end, a verification terminal, a cloud database, and a data cleaning end, wherein the target data selection end is electrically connected to the associated node locking end input node, and the associated node locking end is electrically connected to the source data traceability management end input node, and the source data traceability management end is electrically connected to the cloud database and the verification terminal input node respectively, wherein the cloud database, the verification terminal, and the data cleaning end are electrically connected in sequence from the output node to the input node;

[0033] Among them, this blockchain combines data blocks into a chain data structure in a sequential manner according to time order, which can be understood as multiple nodes with related data circulation. Each node can receive the relevant data transmitted by the previous group of nodes and perform relevant processing, and transmit the processed relevant data to the next node, so as to meet the relevant data needs of several different nodes;

[0034] The target data selection end determines the target data from the node associated with the blockchain based on the verification needs or other needs of the relevant operators, and transmits the determined target data to the associated node locking end, where the target data is determined by the operator himself. Specifically, because some data have related output errors during the processing, when the relevant operators identify such situations, they need to use such data as target data, conduct relevant traceability, lock the source data, complete the source data collection process, and display the collected source data to verify whether there are relevant problems in such source data, and conduct relevant analysis. This application mainly focuses on the source data collection process of relevant target data, and performs relevant data cleaning on the collected source data. After the previous set of data is processed, it will be directly transmitted to the next node, and then the data transmitted to this node from other nodes will be processed;

[0035] The associated node locking end identifies whether similar target data exists at other nodes based on the target data determined at the current moment, and determines similar nodes that are the same as the target node, and transmits the confirmed similar nodes and the target node to the source data traceability management end. The specific method of determining similar nodes is as follows:

[0036] Based on the target data determined at the current moment, determine the target node related to the target data;

[0037] Based on the target node and the target data, identify the previous set of output nodes of the target data, and identify whether the output node sends relevant data to other nodes (that is, other nodes other than the target node). If so, mark the other nodes as the same type of nodes. If not, no marking is performed.

[0038] Specific, combined Figure 2 , the data of the nodes inside the blockchain are all circulated. For example, node A sends relevant data to nodes B and C, and node C sends relevant data to nodes F and G. After node G is marked as the target node, node F belongs to the same type of node, and nodes B and C also belong to the same type of nodes. Node A can be understood as the corresponding master node;

[0039] Among them, the source data tracing management end determines the same type of related data based on the determined target node and the same type of nodes, and performs specific tracing of the source data based on the processing process of the data, and transmits the traced source data to the verification terminal. The tracing process belongs to a related collection process of the corresponding source data;

[0040] Among them, the specific methods for tracing the source data are:

[0041] For the situation where there are similar nodes: based on the determined target node and similar nodes, determine the target data selected by the target node, and then identify the relevant processing rates and the relevant data processed by other similar nodes;

[0042] Mark the current time as T0, determine the real-time processing rate V of the target data corresponding to the target node, and the completed processing volume CL of the corresponding target data, determine the time length of the target node to process the target data based on the completed processing volume CL and the real-time processing rate V, and then determine the time point T1 when the target node receives the initial source data of the target data based on the current time T0 and the time length;

[0043] According to the relevant processing rate and the processed relevant data of other similar nodes, the processing volume of the processed relevant data of the corresponding similar node is determined, and the time length of the corresponding similar node processing the corresponding relevant data is determined in combination with the relevant processing rate, and then based on the current time T0 and the time length, the time point T0 at which the corresponding similar node receives the initial source data of the relevant data is determined. i , where i represents different nodes of the same type;

[0044] Based on different time points T1 and T i, sort the corresponding original source data in order of time to obtain complete original source data;

[0045] Determine the relevant nodes of the initial source data, identify whether the source node of the initial source data sends relevant data to other nodes synchronously, if yes, mark the relevant node as the target node, mark other nodes as similar nodes, and confirm the data according to the processing process of the initial source data of the corresponding target node and the processing process of other data of the same node, and sort the confirmed data in chronological order to obtain the source data, and confirm them in sequence until the relevant node of the determined source data does not send data to other nodes synchronously, and mark the confirmed source data at this moment as the main source data of the target data. For example: based on the previous set of example processes, G is the target node and there is a corresponding target data processing process, then F node also has a set of processing processes, but the order of the two sets of data cannot be confirmed, so proceed Perform time analysis and lock the corresponding receiving time. The data transmission has priority. The data in the front order is transmitted first, and the data in the back order is transmitted second. Then the data in the front order is the front-end data, and the data in the back order is the back-end data. Based on the relevant confirmation of the corresponding time point, the front-end data and the back-end data can be effectively reorganized to obtain the relevant original source data. The original source data is generated by C. Then there is a group of similar nodes B. Then, based on the related processing processes of the similar nodes B and C, the relevant source is traced to confirm the relevant data of the main node A, so as to determine the corresponding main source data. The similar data processed by it are all marked with similar processing, so the related processing process of the B node and the processed data can be directly determined based on the initial source data of C.

[0046] In the case where there is no similar node: determine the original source data of the target data of the target node;

[0047] The relevant nodes of the initial source data identify whether the source nodes of the initial source data send relevant data to other nodes synchronously. If yes, the relevant nodes are marked as target nodes, and other nodes are marked as nodes of the same type. According to the processing process of the initial source data of the corresponding target node and the processing process of other data of the same type of nodes, data confirmation is performed, and the confirmed data is sorted in chronological order to obtain the source data, and confirmation is performed upward in sequence until the relevant nodes of the determined source data do not send data to other nodes synchronously, and the confirmed source data at this moment is marked as the main source data of the target data;

[0048] This source data tracing collection and processing method is faster than the original collection and search method, and its accuracy can also be guaranteed. It is convenient for relevant collection in large-capacity data in large-capacity space, and the data is traced based on the processing process and processing characteristics of different data at its nodes. Then, based on the different time points, the data is reorganized to determine the complete data.

[0049] Collect data in sequence, determine the specific source data of the target data, and make relevant confirmation of the main source data, so as to achieve better collection and processing effects.

[0050] Among them, the main source data will be transmitted to the verification terminal after the relevant traceability collection;

[0051] The verification terminal searches for the data records of the corresponding nodes about the main source data from the cloud database, identifies whether the main source data is accurate, and performs capacity verification. Based on the verification results, it is determined whether it is necessary to perform relevant cleaning of such main source data by the data cleaning terminal. The specific method of verification is as follows:

[0052] Based on the determined capacity Rz of the main source data, and based on the relevant nodes corresponding to the main source data, the data records of the corresponding nodes about the main source data are determined, and from the data records, the original capacity Rs of the data is determined;

[0053] Identify whether Rz and Rs satisfy: (Rz-Rs)>Y1, where Y1 is a preset value, and its specific value is determined by the operator based on experience. If it satisfies, the data cleaning end performs data cleaning on the main source data. If it does not satisfy, the main source data is directly displayed;

[0054] Specifically, when the capacity gap between the data is not large, it means that there are not too many irrelevant data in the corresponding data, and no relevant cleaning is required. When the capacity difference is large, it means that there is a large amount of irrelevant data in the processing of such data, which will cause the irrelevant data to affect the subsequent related analysis results, and it is necessary to clean such main source data.

[0055] The data cleaning end cleans the confirmed main source data, identifies the relevant capacity of irrelevant data based on the relevant capacity difference of the main source data, and analyzes and processes the main source data to identify and remove the irrelevant data. The specific method of analyzing and processing is as follows:

[0056] Based on the data capacity Rz of the main source data and the original capacity Rs, the capacity difference Cz=Rz-Rs is determined;

[0057] Then, the format of the main source data is converted, and its conversion rate is recorded in real time. The recorded conversion rate is subjected to data clustering processing. Based on the clustering results of the data, the conversion rate with abnormal values ​​is determined: a group of parameters with the most conversion rate occurrences is calibrated as standard parameters, and this standard parameter is used as the cluster center point. A set of floating intervals for this standard parameter is constructed, and the floating change value of the floating interval is a preset value, which is formulated by the operator based on experience. The conversion rate that does not belong to this floating interval is calibrated as an abnormal conversion rate;

[0058] Determine the relevant data corresponding to the abnormal conversion rate, and record the data capacity of the corresponding relevant data. If the data capacity = Cz, then directly remove such relevant data from the main source data. If the data capacity ≠ Cz, generate an abnormal acquisition capacity signal and display it. External operators can intervene to remove such irrelevant data. In such abnormal situations, the system cannot identify the corresponding irrelevant data, so human intervention is required to identify other irrelevant data that are obviously abnormal to the original main source data, and remove them one by one to ensure the data quality of the main source data.

[0059] By adopting the format conversion method, after the main source data is traced and collected, the relevant format of this type of data must be unified. When it is converted to other formats, the conversion rate generated is basically the same. When data in other different formats appears, the conversion rate must change significantly. Then, by adopting the data clustering analysis and processing method, such abnormal changes can be found quickly and effectively, and timely processed to lock the abnormal conversion rate, and quickly lock and eliminate irrelevant data to ensure the overall data quality of the main source data.

[0060] Some of the data in the above formulas are dimensionless and numerically calculated. Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0061] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A data collection system based on blockchain, characterized in that: include: The target data selection end determines the target data from the node associated with the blockchain based on the verification requirements or other requirements of the relevant operator, and transmits the determined target data to the associated node locking end, wherein the target data is determined by the operator himself; The associated node locking end identifies whether similar target data exists at other nodes based on the target data determined at the current moment, and determines similar nodes that are the same as the target node, and transmits the confirmed similar nodes and the target node to the source data traceability management end; The source data traceability management terminal determines the same type of related data based on the determined target node and similar nodes, and traces the source data based on the processing process of the data, and transmits the traced main source data to the verification terminal. The specific method is as follows: For the situation where there are similar nodes: based on the determined target node and similar nodes, determine the target data selected by the target node, and then identify the relevant processing rates and the relevant data processed by other similar nodes; Mark the current time as T0, determine the real-time processing rate V of the target data corresponding to the target node, and the completed processing volume CL of the corresponding target data, determine the time length of the target node to process the target data based on the completed processing volume CL and the real-time processing rate V, and then determine the time point T1 when the target node receives the initial source data of the target data based on the current time T0 and the time length; According to the relevant processing rate and the processed relevant data of other similar nodes, the processing volume of the processed relevant data of the corresponding similar node is determined, and the time length of the corresponding similar node processing the corresponding relevant data is determined in combination with the relevant processing rate, and then based on the current time T0 and the time length, the time point T0 at which the corresponding similar node receives the initial source data of the relevant data is determined. i , where i represents different nodes of the same type; Based on different time points T1 and T i , sort the corresponding original source data in order of time to obtain complete original source data; Determine the relevant nodes of the initial source data, identify whether the source node of the initial source data sends relevant data to other nodes synchronously, if yes, mark the relevant node as the target node, mark other nodes as nodes of the same type, and confirm the data according to the processing process of the initial source data of the corresponding target node and the processing process of other data of the same type of node, and sort the confirmed data in chronological order to obtain the source data, and confirm them in turn, until the relevant node of the determined source data does not send data to other nodes synchronously, and mark the confirmed source data at this moment as the main source data of the target data; The verification terminal searches for the data records of the main source data and the corresponding nodes in the cloud database, identifies whether the main source data is accurate, and performs capacity verification. Based on the verification results, it is determined whether the data cleaning terminal needs to perform relevant cleaning on such main source data. The data cleaning end cleans the confirmed main source data, identifies the relevant capacity of irrelevant data based on the relevant capacity difference of the main source data, analyzes and processes the main source data, identifies irrelevant data and removes them.

2. According to claim 1, a data collection system based on blockchain is characterized in that: The specific method of determining the same type of nodes by the associated node locking end is as follows: Based on the target data determined at the current moment, determine the target node related to the target data; Based on the target node and the target data, identify the previous set of output nodes of the target data, and identify whether the output node sends relevant data to other nodes. If so, mark the other nodes as the same type of nodes. If not, no marking is performed.

3. According to the data collection system based on blockchain in claim 1, it is characterized in that: The specific method of tracing and collecting the main source data at the source data traceability management end also includes: In the case where there is no similar node: determine the original source data of the target data of the target node; The relevant nodes of the initial source data identify whether the source nodes of the initial source data send relevant data to other nodes synchronously. If so, the relevant nodes are marked as target nodes, and other nodes are marked as similar nodes. According to the processing process of the initial source data of the corresponding target node and the processing process of other data of the similar node, data confirmation is performed, and the confirmed data is sorted in chronological order to obtain the source data, and confirmation is performed upward in sequence until the relevant nodes of the determined source data do not send data to other nodes synchronously, and the source data confirmed at this moment is marked as the main source data of the target data.

4. A blockchain-based data collection system according to claim 1, characterized in that: The specific method of the verification terminal for performing capacity verification is as follows: Based on the determined capacity Rz of the main source data, and based on the relevant nodes corresponding to the main source data, the data records of the relevant nodes about the main source data are determined, and from the data records, the original capacity Rs of the data is determined; Identify whether Rz and Rs satisfy: (Rz-Rs)>Y1, where Y1 is a preset value. If so, the data cleaning end performs data cleaning on the main source data. If not, the main source data is directly displayed.

5. A blockchain-based data collection system according to claim 4, characterized in that: The specific method of analyzing and processing the main source data at the data cleaning end is as follows: Based on the data capacity Rz of the main source data and the original capacity Rs, the capacity difference Cz=Rz-Rs is determined; Then, the format of the main source data is converted, and its conversion rate is recorded in real time. The recorded conversion rate is subjected to data clustering processing. Based on the clustering results of the data, the conversion rate with abnormal values ​​is determined: a group of parameters with the most conversion rate occurrences is calibrated as standard parameters, and this standard parameter is used as the cluster center point. A group of floating intervals about this standard parameter is constructed, and the floating change value of the floating interval is a preset value. The conversion rate that does not belong to this floating interval is calibrated as an abnormal conversion rate; Determine the relevant data corresponding to the abnormal conversion rate, and record the data capacity of the corresponding relevant data. If the data capacity = Cz, then directly remove such relevant data from the main source data. If the data capacity ≠ Cz, then generate an abnormal acquisition capacity signal and display it.

Images