A data standardization method and system based on cloud blockchain technology

Abstract

The present invention discloses a data standardization method and system based on cloud blockchain technology, including data standard formulation, data standard approval, data standard countersignature, data standard modification, data standard confirmation, data standard sharing and data standard application; and a system applying the standardization method, including a data input port, a data standard module, a data dictionary, a data model, a data quality module, and a data assessment module. The data standardization method is coordinated with blockchain technology.

Description

Technical Field

[0001] The present invention relates to the technical field of data standardization, and in particular to a data standardization method and system based on cloud blockchain technology. Background Art

[0002] With the advent of the big data era, the value of data has been widely recognized by people, and the importance of data has been raised to an unprecedented level. Data is becoming the most important strategic asset in the digital economy era. Scientific data management can turn data into a production factor. Formulating data standards is the premise and important means of data management. However, at present, each unit defines the content of its own standards, and repeatedly invests a lot of manpower, material and financial resources. The level of standard formulation, standard management and standard implementation of each unit is uneven, and the fragmented data standards seriously hinder the data sharing capabilities among units, and cannot give full play to the value of data. Summary of the invention

[0003] The present invention provides a data standardization method and system based on cloud blockchain technology, which can improve data standard management capabilities, circulation capabilities, and implementation capabilities, and improve data quality.

[0004] The technical solution of the present invention is as follows:

[0005] A data standardization method based on cloud blockchain technology, including the formulation, approval, countersignature, modification, confirmation, sharing and application of standards.

[0006] Furthermore, a standards committee composed of multiple users will serve as the main management user of the standards and formulate the standards, and update the data standards based on market changes or changes in customer needs in international, national, regional, industrial, local and other fields.

[0007] Furthermore, by decomposing the standards, a preliminary analysis of the standard content is completed, and the standard English name and English abbreviation are automatically supplemented. The data type of the standard is preliminarily determined based on the decomposed data elements, and the standard operation and maintenance personnel make adjustments based on actual conditions and submit them for approval.

[0008] Furthermore, the standard management user will also receive a standard co-signature notification. After all standard management users have reviewed and approved it, the standard change is approved and enters the pending release state. The standard will take effect after the operation and maintenance personnel release it. If any management user fails to pass the review, the standard application will be returned to the standard operation and maintenance personnel for further adjustment.

[0009] Furthermore, standard users can migrate data standards from the public network environment to their own system environment through the Internet or files. At the same time, users can upload standards defined by their own units. The platform collects standards from various users, compares differences, and forms a standard knowledge base as the basis for subsequent standard updates.

[0010] Further, complete the data standard chain operation, as follows: First: use the client account to deploy smart contracts, use SDK to realize data interaction with the blockchain network, the client initiates a transaction, calls SDK to submit a transaction request, and the endorsing peer endorsement node verifies ① whether the request format is correct ② whether the transaction has been submitted ③ whether the signature is valid ④ whether the initiator has the authority to initiate the corresponding request, and simulates the execution of the transaction, generates the read set and write set for update and makes an endorsement signature, and returns the response to the client. Second: Further, the client responds and queries the request result and the endorsement information of the query response to decide whether to submit the update. The client submits the collected endorsement response and request to the ordering server sorting node, and the ordering server sorts the collected requests by channel and forms blocks according to the block strategy. Third: Further, the odering server broadcasts the block to all peers of the corresponding channel, and the peer independently checks whether each transaction in the block is valid and marks it accordingly. The peer checks whether the content of the transaction complies with the endorsement policy. If so, the peer adds the block to the end of the local blockchain, updates the write set in the valid transaction to the state database, and notifies the client.

[0011] Furthermore, the method of applying the data standard is:

[0012] Use the Chinese name of the object to be standardized to perform a complete keyword search in the term library. If the term is found, it will be returned. The English abbreviation and domain (including data type, length, and precision) of the standard term will be used to replace the object to be standardized and filled into the model tool.

[0013] If the above search fails to match, use the Chinese name of the object to be standardized to perform a complete keyword search for the alias in the term library. If found, the standard term corresponding to the alias is returned, and the Chinese name, English abbreviation and domain of the standard term are used to replace the object to be standardized and filled into the model tool.

[0014] If the above search fails to match successfully, the Chinese name of the object to be standardized is segmented using the standard words and non-standard word sets in the standard word library, and a word list is obtained after segmentation. If the word list is entirely contained in the standard words and non-standard word sets, it is considered that all the word segmentation is successful. If the word list contains non-standard words, they need to be converted into corresponding standard words. After the conversion is completed, the order of combination is replaced based on the words in the word list to obtain terms in which all words are combined in different orders. Using the terms in different order combinations mentioned above, a complete keyword search is performed in the standard term library and aliases. If the search is found, the standard term is returned, and the Chinese name, English abbreviation and domain of the standard term are used to replace the object to be standardized and filled into the model tool.

[0015] If the above search fails to match successfully, the words not included in the word library are removed from the word list obtained by the word segmentation, and after calculating all standard word combinations, fuzzy matching is performed in the standard term library, and the fuzzy matching uses standard words as the minimum complete matching unit. The matching results are scored, and the scoring basis is cumulatively calculated based on the number of standard words contained in the searched terms, that is, the scores of more standard words are high, and finally the weight calculation score is given based on the matching situation. When calculating the weight value, the fuzzy match of the combination word is higher than the sum of all standard word matches. Finally, the scoring result list is output, and the standard term with the highest score is used as the default matching result, and other standard terms with low scores are used as alternative results. In order to improve the intelligent matching degree, the words not included in the standard word library can be output as missing words to supplement the word library of the data standard knowledge base.

[0016] The present invention intelligently recommends and references standard items in model design, and realizes automatic bid-setting in the model design stage. After the model design is completed, the logical model and physical model can be automatically checked, and a review report can be generated with one click, including the bid-setting check of the model, definition completeness check, similarity check of entities, identifiers, primary keys, etc., isolation analysis check, physical information standardization check, etc.

[0017] The present invention also includes a system for applying the above data standardization method, including a data input port, a data standard module, a data dictionary, a data model, a data quality module, and a data assessment module. The data input port inputs data into the system by autonomous collection, manual input, etc.; the data standard module is used in a public network environment, combined with blockchain technology, to provide open, transparent, tamper-proof, and multi-party consensus business standards, technical standards, indicator standards, business terms, and other content formulation and release; the data dictionary module is used in the network environment where the enterprise is located, and the library table structure of the specified database is obtained through the collection tool. Through system processing and manual data supplementation, the corresponding database data dictionary can be established, and the data dictionary and data standard mapping relationship can be achieved. Preliminary standard implementation; the data model module provides a visual modeling tool that automatically applies existing standards to achieve standard implementation while modeling data; the data quality module is used in the network environment where the enterprise is located, and the data quality of the specified data in the specified database is checked by customizing the verification content. For the data quality problems found, it can be distributed to the relevant responsible persons for problem repair; the data assessment module can customize the assessment plan, and the data quality improvement purpose is achieved by assessing the generation and repair of data quality problems of unit staff through rewards and punishments.

[0018] The beneficial effects brought by the technical solution provided by the present invention are:

[0019] Data standards are important basic tools in the digital age and an important foundation for promoting the construction of the data factor market. The platform adheres to the open principle of "one-time construction and repeated use", adopts the operating mechanism of "open compatibility, cooperative sharing, and keeping pace with the times", and constantly adapts to the needs of developing data standards work. In an open way, it gathers the wisdom of the people, and innovates and formulates high-quality data standards that are versatile, easy to implement, flexible, and easy to use in a timely manner.

[0020] Ensure the consistency of terms and domains in the process of data use and reduce the manual error rate. Inconsistency in design specifications between various systems in the process of enterprise information construction will lead to increased process costs such as communication, conversion, and cleaning in the process of data integration. At the same time, in the process of data model design, in order to ensure the consistency of terms and domains, it is necessary to rely on the construction of an enterprise-level unified data standard knowledge base and the use of automated application methods in the data model design stage. Therefore, in the technical solution provided by the present invention, on the basis of establishing a data standard knowledge base, it is applied through automated means to avoid the error rate of manual conversion. Improve the efficiency of manual selection and matching and reduce manpower input. The unification of terms and domains in the process of data model design is not a one-time solution. Even if it is applied for the first time through partial automation, in the process of continuous expansion of system functions, new additions and changes still need to be paid attention to. Therefore, the present invention includes the feature of automatic association of the data standard knowledge base, which not only effectively reduces manpower input during the first application, but also plays a role in improving efficiency during continuous maintenance. The present invention combines the process of standard formulation and use with blockchain technology, and utilizes the characteristics of blockchain such as openness, transparency, tamper-proof, and multi-party consensus. Through the storage and traceability of the formulation, approval, modification, confirmation, sharing, and application of data standards, the legality and compliance of data standard formulation and application are guaranteed in key processes such as the authority and control of data standard approval, the confirmation and sharing of data standards, and the audit and supervision of data standards, so as to improve the consistency of enterprises' understanding of professional terms and communication efficiency. The construction of an enterprise-level data standard knowledge base plays a role in unifying business caliber and data interaction caliber, avoiding the communication efficiency in the process of system integration and data exchange, and also reducing the difficulty and error rate of integration and interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 This is a flow chart of a standard release according to an embodiment of the present invention;

[0023] Figure 2 It is a framework diagram of the combination of data standard operation and management and blockchain in the present invention;

[0024] Figure 3 It is a schematic diagram of the overall process of the intelligent model design method based on the data standard knowledge base of the present invention;

[0025] Figure 4It is a detailed flow chart of the intelligent standardization model design of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0027] Embodiment 1:

[0028] In the logical model of the data model design tool, select the attributes that need to be standardized as the objects to be standardized. According to the words, domains, and terms in the custom data standard knowledge base, the steps for standard synchronization of the objects to be standardized within the selected range are as follows:

[0029] Use the Chinese name of the object to be standardized to perform a complete keyword search in the term library. If the search is found, the term is returned and the English abbreviation and domain (including data type, length, and precision) of the standard term are used to replace the object to be standardized. For example, if the object to be standardized is "email", and the standard term "email" is searched in the standard term library, the English abbreviation, data type, and length of "email" in the standard term library are used to fill in the model tool.

[0030] If no match is found, the Chinese name of the object to be standardized is used to perform a complete keyword search for the alias in the term library. If found, the standard term corresponding to the alias is returned, and the Chinese name, English abbreviation and domain of the standard term are used to replace the object to be standardized. For example, if the object to be standardized is "Email", and the standard term library searches for "Email" as an alias of the standard term "email", then the object to be standardized is replaced with "email" in the standard term library, and the English abbreviation, data type and length of "email" are filled in the model tool.

[0031] If the match is not successful, the Chinese name of the standardized object is segmented using the standard words and non-standard word sets in the standard word library, and a word list after segmentation is obtained. If the word list is all included in the standard words and non-standard word sets, it is considered that all the segmentation is successful. If the word list contains non-standard words, it needs to be converted into corresponding standard words. After the conversion is completed, the combination order is replaced based on the words in the word list to obtain the terms in which all words are combined in different orders. For example, when the word to be standardized is "employee entry number", the standard word library contains three words "employee", "entry", and "number". At the same time, "employee" in the non-standard word library corresponds to the standard word "employee". "Employee", "entry", "number" and "employee" are used to segment the standardized words, and the segmentation results are three words "employee + entry + number". After replacing the non-standard words, "employee + entry + number" is obtained. After replacing the combination order, multiple terms consisting of standard words such as "employee entry number", "employee number entry", "entry employee number", "entry number employee", "number entry employee" and "number employee entry" are obtained.

[0032] All the combined terms obtained by using the above segmentation are fully searched in the term library and aliases of the standard knowledge base. For example, if the standard term "employee number" is included in the standard term library, when the term is fully matched, the standard term is searched in the term library of the standard knowledge base, and "employee number" is replaced with "employee number", and the English abbreviation, data type and length of "employee number" are filled into the model tool.

[0033] If the match is not successful, the words not included in the word library are removed from the word list, and then the score calculation recommendation is performed, and the words not included in the standard word library are output as missing words. For example, the object to be standardized "Securities Code Information" is processed through multiple steps to obtain the three words "Securities + Code + Information". Since "Information" is not included in the standard word library and the non-standard word library, it is removed from the combination and output as a missing word, that is, the two standard words "Securities + Code" are used as input for the next step.

[0034] According to the word list obtained by the above segmentation, after calculating all standard word combinations, fuzzy matching is performed in the standard term library, and the fuzzy matching uses standard words as the minimum complete matching unit. The matching results are scored, and the scoring basis is the cumulative calculation of the number of standard words contained in the searched terms, that is, the higher the score is, the more standard words are contained. For example, when the two standard words "Securities + Code" are entered, all combinations of these two words are first constructed to obtain "Securities", "Code", "Securities Code" and "Code Securities", among which "Securities" and "Code" belong to the original standard words, and "Securities Code" and "Code Securities" belong to the combination of the original words. Use these four words to search in the standard term library. If the standard term library contains "Listed Securities Code", the fuzzy matching of the combination word is successful. If the standard term library contains "Listed Securities Unique Code", the fuzzy matching of the combination word fails, but the standard word matching is successful, and it contains the two standard words "Securities" and "Code". If the standard term library contains "Listed Securities Name", the fuzzy matching of the combination word fails, but the standard word matching is successful, and it contains only one standard word. Finally, the weighted score is calculated based on the matching situation. When calculating the weight value, the fuzzy match of the combination word is higher than the sum of all the successful matches of the standard words. In the standard word matching results, the calculation is based on the number of standard words included. According to the above calculation, the number of standard words entered is 2, and the combination word matching result is greater than 2. For multiple combination words that are successfully matched, the score of the positive combination result is calculated to be greater than the reverse combination score, that is, the score of "Securities Code" is higher than that of "Code Securities", and the combination word matching results are all greater than the scores of the standard word matching results "Listed Securities Unique Code" and "Listed Securities Name". The final scoring results are output in a list, with the highest-scoring standard terms as the default matching results, and other low-scoring standard terms as alternative results. After the model design and standard automatic reference are completed, the model is quality checked, including the model's label check, definition completeness check, entity, identifier, primary key and other similarity checks, isolation analysis check, physical information standardization check, etc., and the model inspection report is automatically generated.

[0035] Embodiment 2:

[0036] A system applying the above data standardization method includes a data input port, a data standard module, a data dictionary, a data model, a data quality module, and a data assessment module. The data input port is a computer and a device that can autonomously identify user preset data. Data is input into the system through autonomous collection, manual input, etc. The data standard module is used in a public network environment and combined with blockchain technology to provide the formulation and release of open, transparent, tamper-proof, and multi-party consensus business standards, technical standards, indicator standards, business terms, etc. The data dictionary module is used in the network environment where the enterprise is located. The library table structure of the specified database is obtained through the collection tool. Through system processing and manual data supplementation, the corresponding database data dictionary can be established. The mapping relationship between the data dictionary and the data standard can achieve preliminary standard implementation; the data model module provides a visual modeling tool that automatically applies existing standards to achieve standard implementation while modeling data; the data quality module is used in the network environment where the enterprise is located. Through customized verification content, the data quality of the specified data in the specified database is checked. For the data quality problems found, they can be distributed to the relevant responsible persons for problem repair; the data assessment module can customize the assessment plan, and the purpose of improving data quality is achieved through rewards and punishments by assessing the generation and repair of data quality problems of unit staff.

Claims

1. A data standardization method based on cloud blockchain technology, including data standard formulation, data standard approval, data standard countersignature, data standard modification, data standard confirmation, data standard sharing and data standard application, with the specific steps of S1: users formulate standards and update data standards according to market changes or customer demand changes in international, national, regional, industrial and local fields; S2: complete the preliminary analysis of the standard content through standard word decomposition, automatically complete the supplement of the standard English name and English abbreviation, preliminarily determine the data type of the standard according to the decomposed data element, and submit it for approval after adjustment according to the actual situation; S3: after the user receives the standard countersignature notification, all users have reviewed and approved it, the standard change is passed and enters the pending release state, and the standard takes effect after the operation and maintenance personnel release it; if there is a management user who fails to review it, the standard application is returned to the standard operation and maintenance personnel for further adjustment; S4: users migrate the data standards in the public network environment to the system environment of the unit through the network or file, and at the same time, users upload the standards defined by the unit, and the platform collects the standards of each user, compares the differences, and forms a standard knowledge base as the basis for subsequent standard updates; S5: data standard sharing and application, characterized in that, It also includes standard data on-chain operations, as follows: First: Use the client account to deploy smart contracts, use the SDK to implement data interaction with the blockchain network, the client initiates a transaction, calls the SDK to submit a transaction request, and the endorsing peer verifies ① whether the request format is correct ② whether the transaction has been submitted ③ whether the signature is valid ④ whether the initiator has the authority to initiate the corresponding request, and simulates the execution of the transaction, generates the read set and write set for update, and makes an endorsement signature, and returns the response to the client; Second: The client responds and queries the request result and the endorsement information of the query response to decide whether to submit the update; the client submits the collected endorsement response and request to the ordering server sorting node, and the ordering server sorts the collected requests by channel and forms blocks according to the block strategy; Third: The odering server broadcasts the block to all peers of the corresponding channel. The peer independently checks whether each transaction in the block is valid and marks it accordingly. The peer checks whether the content of the transaction complies with the endorsement policy. If it does, the peer block is added to the end of the local blockchain, and the write set in the valid transaction is updated to the state database, and the client is notified at the same time. The steps of applying the data standard are: S1: Use the Chinese name of the object to be standardized and conduct a complete keyword search in the terminology database. If the term is found, the term is returned and the English abbreviation and domain of the standard term are used to replace the object to be standardized and filled into the model tool; S2: If the above search fails to match, use the Chinese name of the object to be standardized and perform a complete keyword search for the alias in the term library. If found, return the standard term corresponding to the alias, use the Chinese name, English abbreviation and domain of the standard term to replace the object to be standardized, and fill it into the model tool; S3: If the above search fails to match, the Chinese name of the object to be standardized is segmented using the standard words and non-standard word set in the standard word library, and a word list after segmentation is obtained. If the word list is fully contained in the standard words and non-standard word set, it is considered that all the word segmentation is successful. If the word list contains non-standard words, they need to be converted into corresponding standard words. After the conversion is completed, the order of the combination is permuted based on the words in the word list to obtain the terms in which all the words are combined in different orders; S4: Use the terms in different order combinations to conduct a complete keyword search in the standard terminology database and aliases. If found, the standard terminology is returned, and the Chinese name, English abbreviation and domain of the standard terminology are used to replace the object to be standardized and filled into the model tool; S5: If the above search fails to match, remove the words not included in the word library from the word list obtained by the word segmentation, calculate all standard word combinations, and then perform fuzzy matching in the standard term library, with the standard word being the minimum complete matching unit; The matching results are scored. The scoring basis is the cumulative calculation of the number of standard words contained in the searched terms, that is, the more standard words are contained, the higher the score is. Finally, the score is calculated based on the weight of the matching situation. When calculating the weight value, the fuzzy match of the combination word is higher than the sum of the successful matches of all standard words. Finally, a list of scoring results is output, with the standard term with the highest score as the default matching result, and other standard terms with low scores as alternative results.

2. An application data standardization system, including a data input port, a data standard module, a data dictionary module, a data model module, a data quality module, and a data assessment module; the data input port inputs data into the system by autonomous collection and manual input; the data standard module is used in a public network environment, combined with blockchain technology, to provide open, transparent, tamper-proof, and multi-party consensus business standards, technical standards, indicator standards, and the formulation and release of business terminology content; the data dictionary module is used in the network environment where the enterprise is located, and obtains the library table structure of the specified database through a collection tool, and establishes a corresponding database data dictionary through system processing and manual data supplementation, and the mapping relationship between the data dictionary and the data standard is achieved to achieve preliminary standard implementation; the data model module provides a visual modeling tool, which automatically applies existing standards to achieve standard implementation while modeling data; the data quality module is used in the network environment where the enterprise is located, and checks the data quality of the specified data in the specified database through customized inspection content, and distributes the data quality problems found to the relevant responsible persons for problem repair; the data assessment module customizes the assessment plan, and assesses the generation and repair of data quality problems of unit staff, and achieves the purpose of improving data quality through rewards and punishments, which is characterized by: The data standard module is used in the method described in claim 1.

Images