A data asset management evaluation system and method based on data governance

Abstract

The present invention discloses a data asset management evaluation system and method based on data governance, and relates to the technical field of data analysis. The present invention uses an authority management port to access a target database, traverses the target database storage data by setting up a data retrieval channel, and outputs the traversed data; assigns a number to the database traversal output data based on the associated attributes of the corresponding data, and constructs a time series data set through data integration processing; retrieves the call record data of the corresponding data in each time series data set to perform data call quality analysis on the data in each time series data set; performs a field contribution rate analysis on the time series data set corresponding to each type of data based on the analysis result, and performs an association analysis on the data in each field in combination with the analysis data to obtain the associated data chain of the data in each field; and according to the result of the field data association analysis, constructs, stores and outputs the associated data chain of the corresponding field data in the database.

Description

Technical Field

[0001] The present invention relates to the technical field of data analysis, and in particular to a data asset management evaluation system and method based on data governance. Background Art

[0002] Data asset management refers to the systematic management and control of data assets within an organization, aiming to maximize the value of data, ensure the quality and security of data, and promote the effective use of data; data governance is a collection of policies, processes, roles and responsibilities for data asset management and control, which ensures the quality and security of data throughout its life cycle and improves the quality and value of data.

[0003] In today's environment, a large amount of personal data or corporate data is generated every moment. Traditionally, data is processed by manual judgment and selective storage. In today's big data era, there are more advanced data storage modes, which greatly improve the storage efficiency of traditional data and realize diverse storage of data by setting block classification. However, this mode only meets people's needs for data storage, but it is still inefficient in terms of how to efficiently use the stored data and how to connect the correlation between data. Users can only search by themselves according to their needs, or gradually filter by keywords. This mode has defects in the efficient use of data and the improvement of its value. Summary of the invention

[0004] The purpose of the present invention is to provide a data asset management evaluation system and method based on data governance to solve the problems raised in the prior art.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A data asset management evaluation method based on data governance, the method comprising the following steps:

[0007] S100, using the authority management port to access the target database, traversing the data stored in the target database by setting up a data retrieval channel, and outputting the traversed data;

[0008] S200, assigning numbers to the output data of the database traversal based on the associated attributes of the corresponding data, and constructing a time series data set through data integration processing;

[0009] S300, retrieve the call record data of the corresponding data in each time series data set to perform data call quality analysis on the data in each time series data set; based on the analysis results, perform field contribution rate analysis on the time series data set corresponding to each type of data, and perform association analysis on the data in each field in combination with the analysis data to obtain the associated data chain of the data in each field;

[0010] S400: According to the domain data association analysis result, the association data chain corresponding to the domain data in the database is constructed, stored and output.

[0011] The specific steps of S100 using the authority management port to access the target database, traversing the target database storage data by setting up a data retrieval channel, and outputting the traversed data are as follows:

[0012] S101, access the target database through the access port, using the authority administrator identity, and construct a temporary data retrieval channel for the database; the target database may be an enterprise database or a personal database, etc.;

[0013] S102, traversing the stored data in the database in its entirety, traversing the stored data in the database through a fixed traversal window until the stored data in the database is completely traversed, and outputting the traversed data for analysis.

[0014] The specific steps of S200 for assigning numbers to the database traversal output data based on the associated attributes of the corresponding data and constructing a time series data set through data integration processing are as follows:

[0015] S201, extracting corresponding associated data from the traversed output data in the database; the associated data is the domain data, type data and storage time data of the target data; assigning numbers based on the associated data features of each data, and constructing feature index numbers for each data;

[0016] S202, based on the characteristic index number of each data, classify and process the data based on the field and type to which the data belongs; perform time series integration processing on the data of corresponding types in each field after processing, integrate and coordinate the data of the same type in the same field and construct a set, perform time series arrangement processing on the data in the set based on its storage time characteristics, and construct a time series data set of the data of the corresponding type in the corresponding field; wherein the time series arrangement processing takes the storage time of each data as the comparison analysis object, and sorts the set in the order of storage time.

[0017] The S300 retrieves the call record data of the corresponding data in each time series data set to perform data call quality analysis on the data in each time series data set; based on the analysis results, the field contribution rate analysis is performed on the time series data set corresponding to each type of data, and the correlation analysis is performed on the data in each field in combination with the analysis data. The specific steps of obtaining the correlation data chain of the data in each field are as follows:

[0018] S301. Based on the corresponding time series data sets of each type in each field, extract the data of each time point in each time series data set respectively; perform call record data query on each extracted data respectively; the call record is the target data stored in the database from the corresponding time point to the current real-time time, the call time and the call number; perform real-time data quality assessment on each target data according to the call data of each target data, and the analysis formula is:

[0019]

[0020] Where Dq(f,p,n) is the real-time data quality evaluation value of the data of element number n in the set corresponding to field number f, type number p; α and β are evaluation coefficients; m(f,p,n) and h(f,p,n) are the call times and call time of the data of element number n in the set corresponding to field number f, type number p; T is the current real-time time; t(f,p,n) is the storage time of the data of element number n in the set corresponding to field number f, type number p;

[0021] According to the real-time data quality assessment analysis results of each data in each time series data set, a comprehensive data quality assessment is performed on each type of corresponding time series data set. The calculation formula is:

[0022]

[0023] Among them, Cq(f,p) is the comprehensive type data quality assessment value of the corresponding field number f and the type data time series set number p; based on the comprehensive type data quality assessment data of each type of time series data set in the corresponding field, the field data value contribution analysis is performed on the current type of time series data, and its calculation formula is:

[0024]

[0025] Among them, Cv(f,p) is the domain data value contribution value of the time series set of data of type p corresponding to domain f; A(f,p) is the domain contribution impact ratio of data of type p corresponding to domain f; Cq(f,p) max and Cq(f,p) min are the maximum and minimum values ​​of the comprehensive data quality assessment values ​​of each type of data time series set in the corresponding field number f;

[0026] S302. Combine the quality assessment data and value contribution analysis data corresponding to each type of data in each field, and perform association analysis on the data in each type of time series data in each field; construct feature coordinates for the data in each type of time series data in each field, respectively, and use the field identity feature index of the corresponding data, the type identity feature index, the actual data volume of the corresponding data, and the storage time of the data as dimension data to construct feature coordinates; wherein the field identity feature index and the type identity feature index are the identity feature code data of the field and type to which the corresponding data belongs, respectively, and are used to represent the identity information of the corresponding field and type; according to the feature coordinate data of the data in each type of time series data in each field, perform traversal association analysis on the data in each type of time series data in each field, and the calculation formula is:

[0027]

[0028] Among them, m1 and m2 are any two data traversed in the corresponding types of time series data in each field; R(m1, m2) is the data association value between data m1 and m2; m1→Cv(f, p) and m1→Cq(f, p) are the domain data value contribution value and the comprehensive type data quality assessment value of the data time series set corresponding to data m1; m2→Cv(f, p) and m2→Cq(f, p) are the domain data value contribution value and the comprehensive type data quality assessment value of the data time series set corresponding to data m2; Z(m1) and Z(m2) are the derived feature vectors of the feature coordinates corresponding to data m1 and m2 respectively; among them, the derived feature vector of the data feature coordinates is constructed by using the data feature coordinates and the coordinate origin; the calculation between vectors is the vector dot multiplication;

[0029] By setting the data association threshold R(x), the association between the traversed data is judged by comparing the threshold; if R(m1,m2)<R(x), the two data currently traversed are judged to be weakly associated; if R(m1,m2)≥R(x), the two data currently traversed are judged to be strongly associated; according to the judgment result, by completely traversing the data and performing association analysis, the data that has a strong association with the target data is annotated and extracted, and through the association analysis results, the data chain is connected in sequence from strong to weak to construct the associated data chain of the target data.

[0030] The specific steps of constructing, storing and outputting the associated data chain of the corresponding domain data in the database according to the domain data association analysis result in S400 are as follows:

[0031] S401, recording and storing the domain type time series data set division of the data stored in the database and the quality assessment and domain contribution analysis data of the corresponding data time series set;

[0032] S402: Feedback and output the associated analysis results of each data in the database and the associated data chain constructed by the corresponding associated data.

[0033] A data asset management and evaluation system based on data governance, the system comprising a data access module, a data set construction module, a data association evaluation module and a data output module;

[0034] The data access module uses the authority management port to access the target database, traverses the target database storage data by setting up a data retrieval channel, and outputs the traversed data; the data set construction module assigns numbers to the database traversal output data based on the associated attributes of the corresponding data, and constructs a time series data set through data integration processing; the data association evaluation module retrieves the call record data of the corresponding data in each time series data set to perform data call quality analysis on the data in each time series data set; based on the analysis results, the field contribution rate of the corresponding time series data set of each type of data is analyzed, and the association analysis of the data in each field is performed in combination with the analysis data to obtain the associated data chain of the data in each field; the data output module constructs, stores and outputs the associated data chain of the corresponding field data in the database according to the results of the field data association analysis.

[0035] The data access module includes a database access unit and a data retrieval unit;

[0036] The database access unit accesses the target database through the access port of the target database using the identity of the authority administrator, and constructs a temporary data retrieval channel for the database;

[0037] The data retrieval unit performs a full traversal of the stored data in the database, traverses the stored data in the database through a fixed traversal window, until the stored data in the database is completely traversed, and outputs the traversed data for analysis.

[0038] The data set construction module includes a data processing unit and a time series data set construction unit;

[0039] The data processing unit extracts the corresponding associated data from the traversed output data in the database; the associated data is the domain data, type data and storage time data of the target data; assigns numbers based on the associated data features of each data, and constructs the feature index number of each data;

[0040] The time series data set construction unit performs classification processing based on the feature index number of each data and based on the field and type to which the data belongs; performs time series integration processing on the corresponding types of data in each field after processing, integrates and coordinates the same type of data in the same field and constructs a set, performs time series arrangement processing on the data in the set based on its storage time characteristics, and constructs a time series data set of corresponding types of data in the corresponding field.

[0041] The data association evaluation module includes a data comprehensive evaluation unit and a data association analysis unit;

[0042] The data comprehensive evaluation unit extracts data at each time point in each time series data set based on the corresponding time series data sets of each type in each field; performs call record data query on each extracted data; the call record is the target data stored in the database from the corresponding time point to the current real-time time, the call time and the number of calls; performs real-time data quality evaluation on each target data according to the call data of each target data; performs comprehensive type data quality evaluation on each type of corresponding time series data set according to the real-time data quality evaluation analysis results of each data in each time series data set; performs field data value contribution analysis on the current type of time series data based on the comprehensive type data quality evaluation data of each type of time series data set in the corresponding field;

[0043] The data association analysis unit combines the quality assessment data and value contribution analysis data corresponding to each type of data in each field, and performs association analysis on the data in the corresponding types of time series data in each field; constructs feature coordinates for the data in the corresponding types of time series data in each field, respectively, and constructs feature coordinates with the field identity feature index of the corresponding data, the type identity feature index, the actual data volume of the corresponding data and the storage time of the data as dimension data; performs traversal association analysis on the data in the corresponding types of time series data in each field according to the feature coordinate data of the data in the corresponding types of time series data in each field; sets a data association threshold, and judges the association between the traversed data by comparing the threshold; and according to the judgment result, constructs an associated data chain of the target data by completely traversing the data and performing association analysis, and by the association analysis result.

[0044] The data output module includes a data recording unit and a data link output unit;

[0045] The data recording unit records and stores the domain type time series data set division of the data stored in the database and the quality assessment and domain contribution analysis data of the corresponding data time series set;

[0046] The data chain output unit feeds back and outputs the associated analysis results of each data in the database and the associated data chain constructed by the corresponding associated data.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] The present invention retrieves data stored in a personal or corporate database by serially connecting the database, and optimizes and classifies the retrieved data through a series of data processing methods to construct a corresponding time series data set; performs real-time data quality assessment on each data based on the time series data set, and comprehensively judges the comprehensive value contribution of the time series data set; performs association analysis on any data in the database in combination with the quality analysis and value contribution analysis of each data, and realizes data serial connection of highly correlated data to construct an associated data chain based on the association analysis results; the present invention can effectively analyze the real-time value status of each data in the database, and associate and combine based on the value status of each data, autonomously and intelligently perform effective serial output on the relevant data, greatly improve the utilization efficiency of the data, and improve the use value of the serial data of each data through the associated data chain. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a structural schematic diagram of a data asset management and evaluation system based on data governance in the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] Example: Figure 1 As shown, the present invention provides a technical solution:

[0052] A data asset management evaluation method based on data governance, the method comprising the following steps:

[0053] S100, using the authority management port to access the target database, traversing the data stored in the target database by setting up a data retrieval channel, and outputting the traversed data;

[0054] S200, assigning numbers to the output data of the database traversal based on the associated attributes of the corresponding data, and constructing a time series data set through data integration processing;

[0055] S300, retrieve the call record data of the corresponding data in each time series data set to perform data call quality analysis on the data in each time series data set; based on the analysis results, perform field contribution rate analysis on the time series data set corresponding to each type of data, and perform association analysis on the data in each field in combination with the analysis data to obtain the associated data chain of the data in each field;

[0056] S400: According to the domain data association analysis result, the association data chain corresponding to the domain data in the database is constructed, stored and output.

[0057] The specific steps of S100 using the authority management port to access the target database, traversing the target database storage data by setting up a data retrieval channel, and outputting the traversed data are as follows:

[0058] S101, access the target database through the access port using the authority administrator, and build a temporary data retrieval channel for the database;

[0059] S102, traversing the stored data in the database in its entirety, traversing the stored data in the database through a fixed traversal window until the stored data in the database is completely traversed, and outputting the traversed data for analysis.

[0060] The specific steps of S200 for assigning numbers to the database traversal output data based on the associated attributes of the corresponding data and constructing a time series data set through data integration processing are as follows:

[0061] S201, extracting corresponding associated data from the traversed output data in the database; the associated data is the domain data, type data and storage time data of the target data; assigning numbers based on the associated data features of each data, and constructing feature index numbers for each data;

[0062] S202. Based on the characteristic index number of each data, classify and process the data based on the field and type to which the data belongs; perform time series integration processing on the data of corresponding types in each field after processing, integrate and coordinate the data of the same type in the same field and construct a set, perform time series arrangement processing on the data in the set based on its storage time characteristics, and construct a time series data set of the corresponding type of data in the corresponding field.

[0063] The S300 retrieves the call record data of the corresponding data in each time series data set to perform data call quality analysis on the data in each time series data set; based on the analysis results, the field contribution rate analysis is performed on the time series data set corresponding to each type of data, and the correlation analysis is performed on the data in each field in combination with the analysis data. The specific steps of obtaining the correlation data chain of the data in each field are as follows:

[0064] S301. Based on the corresponding time series data sets of each type in each field, extract the data of each time point in each time series data set respectively; perform call record data query on each extracted data respectively; the call record is the target data stored in the database from the corresponding time point to the current real-time time, the call time and the call number; perform real-time data quality assessment on each target data according to the call data of each target data, and the analysis formula is:

[0065]

[0066] Where Dq(f,p,n) is the real-time data quality evaluation value of the data of element number n in the set corresponding to field number f, type number p; α and β are evaluation coefficients; m(f,p,n) and h(f,p,n) are the call times and call time of the data of element number n in the set corresponding to field number f, type number p; T is the current real-time time; t(f,p,n) is the storage time of the data of element number n in the set corresponding to field number f, type number p;

[0067] According to the real-time data quality assessment analysis results of each data in each time series data set, a comprehensive data quality assessment is performed on each type of corresponding time series data set. The calculation formula is:

[0068]

[0069] Among them, Cq(f,p) is the comprehensive type data quality assessment value of the corresponding field number f and the type data time series set number p; based on the comprehensive type data quality assessment data of each type of time series data set in the corresponding field, the field data value contribution analysis is performed on the current type of time series data, and its calculation formula is:

[0070]

[0071] Among them, Cv(f,p) is the domain data value contribution value of the time series set of data of type p corresponding to domain f; A(f,p) is the domain contribution impact ratio of data of type p corresponding to domain f; Cq(f,p) max and Cq(f,p) min are the maximum and minimum values ​​of the comprehensive data quality assessment values ​​of each type of data time series set in the corresponding field number f;

[0072] S302. Combine the quality assessment data and value contribution analysis data corresponding to each type of data in each field, and perform association analysis on the data in each type of time series data in each field; construct feature coordinates for the data in each type of time series data in each field, respectively, and use the field identity feature index, type identity feature index, actual data volume of the corresponding data, and data storage time of the corresponding data as dimension data to construct feature coordinates; perform traversal association analysis on the data in each type of time series data in each field according to the feature coordinate data of the data in each type of time series data in each field, and the calculation formula is:

[0073]

[0074] Among them, m1 and m2 are any two data traversed in the corresponding types of time series data in each field; R(m1, m2) is the data association value between data m1 and m2; m1→Cv(f, p) and m1→Cq(f, p) are the domain data value contribution value and the comprehensive type data quality assessment value of the data time series set corresponding to data m1; m2→Cv(f, p) and m2→Cq(f, p) are the domain data value contribution value and the comprehensive type data quality assessment value of the data time series set corresponding to data m2; Z(m1) and Z(m2) are the derived feature vectors of the feature coordinates corresponding to data m1 and m2 respectively;

[0075] By setting the data association threshold R(x), the association between the traversed data is judged by comparing the threshold; if R(m1,m2)<R(x), the two data currently traversed are judged to be weakly associated; if R(m1,m2)≥R(x), the two data currently traversed are judged to be strongly associated; according to the judgment result, by completely traversing the data and performing association analysis, the data that has a strong association with the target data is annotated and extracted, and through the association analysis results, the data chain is connected in sequence from strong to weak to construct the associated data chain of the target data.

[0076] The specific steps of constructing, storing and outputting the associated data chain of the corresponding domain data in the database according to the domain data association analysis result in S400 are as follows:

[0077] S401, recording and storing the domain type time series data set division of the data stored in the database and the quality assessment and domain contribution analysis data of the corresponding data time series set;

[0078] S402: Feedback and output the associated analysis results of each data in the database and the associated data chain constructed by the corresponding associated data.

[0079] A data asset management and evaluation system based on data governance, the system comprising a data access module, a data set construction module, a data association evaluation module and a data output module;

[0080] The data access module uses the authority management port to access the target database, traverses the target database storage data by setting up a data retrieval channel, and outputs the traversed data; the data set construction module assigns numbers to the database traversal output data based on the associated attributes of the corresponding data, and constructs a time series data set through data integration processing; the data association evaluation module retrieves the call record data of the corresponding data in each time series data set to perform data call quality analysis on the data in each time series data set; based on the analysis results, the field contribution rate of the corresponding time series data set of each type of data is analyzed, and the association analysis of the data in each field is performed in combination with the analysis data to obtain the associated data chain of the data in each field; the data output module constructs, stores and outputs the associated data chain of the corresponding field data in the database according to the results of the field data association analysis.

[0081] The data access module includes a database access unit and a data retrieval unit;

[0082] The database access unit accesses the target database through the access port of the target database using the identity of the authority administrator, and constructs a temporary data retrieval channel for the database;

[0083] The data retrieval unit performs a full traversal of the stored data in the database, traverses the stored data in the database through a fixed traversal window, until the stored data in the database is completely traversed, and outputs the traversed data for analysis.

[0084] The data set construction module includes a data processing unit and a time series data set construction unit;

[0085] The data processing unit extracts the corresponding associated data from the traversed output data in the database; the associated data is the domain data, type data and storage time data of the target data; assigns numbers based on the associated data features of each data, and constructs the feature index number of each data;

[0086] The time series data set construction unit performs classification processing based on the feature index number of each data and based on the field and type to which the data belongs; performs time series integration processing on the corresponding types of data in each field after processing, integrates and coordinates the same type of data in the same field and constructs a set, performs time series arrangement processing on the data in the set based on its storage time characteristics, and constructs a time series data set of corresponding types of data in the corresponding field.

[0087] The data association evaluation module includes a data comprehensive evaluation unit and a data association analysis unit;

[0088] The data comprehensive evaluation unit extracts data at each time point in each time series data set based on the corresponding time series data sets of each type in each field; performs call record data query on each extracted data; the call record is the target data stored in the database from the corresponding time point to the current real-time time, the call time and the number of calls; performs real-time data quality evaluation on each target data according to the call data of each target data; performs comprehensive type data quality evaluation on each type of corresponding time series data set according to the real-time data quality evaluation analysis results of each data in each time series data set; performs field data value contribution analysis on the current type of time series data based on the comprehensive type data quality evaluation data of each type of time series data set in the corresponding field;

[0089] The data association analysis unit combines the quality assessment data and value contribution analysis data corresponding to each type of data in each field, and performs association analysis on the data in the corresponding types of time series data in each field; constructs feature coordinates for the data in the corresponding types of time series data in each field, respectively, and constructs feature coordinates with the field identity feature index of the corresponding data, the type identity feature index, the actual data volume of the corresponding data and the storage time of the data as dimension data; performs traversal association analysis on the data in the corresponding types of time series data in each field according to the feature coordinate data of the data in the corresponding types of time series data in each field; sets a data association threshold, and judges the association between the traversed data by comparing the threshold; and according to the judgment result, constructs an associated data chain of the target data by completely traversing the data and performing association analysis, and by the association analysis result.

[0090] The data output module includes a data recording unit and a data link output unit;

[0091] The data recording unit records and stores the domain type time series data set division of the data stored in the database and the quality assessment and domain contribution analysis data of the corresponding data time series set;

[0092] The data chain output unit feeds back and outputs the association analysis results of each data in the database and the association data chain constructed by the corresponding association data;

[0093] In the example:

[0094] Now a certain enterprise has introduced the data asset management and evaluation system based on data governance of the present invention, and carried out data management evaluation on the stored data in its database; then, the target database is accessed through the access port using the identity of the authorized administrator, and a temporary data retrieval channel is constructed for the database; the stored data is fully traversed in the database, and the stored data in the database is traversed through a fixed traversal window until the database storage data is completely traversed, and the traversed data is output and analyzed; the corresponding associated data is extracted from the traversed output data in the database; the associated data is the field data, type data and storage time data of the target data; numbers are assigned based on the associated data features of each data, and a feature index number of each data is constructed; based on the feature index number of each data, classification processing is carried out based on the field and type to which the data belongs; time series integration processing is carried out on the corresponding types of data in each field after processing, and the same type of data in the same field is integrated and coordinated to construct a set, and the data in the set is arranged in time series based on its storage time features, and a time series data set of corresponding types of data in the corresponding field is constructed;

[0095] Based on the corresponding types of time series data sets in various fields, the data of each time point in each time series data set are extracted respectively; the call record data query is performed on each extracted data respectively; the call record is the target data stored in the database from the corresponding time point to the current real-time time, the call time and the number of calls; the real-time data quality assessment of each target data is performed according to the call data of each target data, and the analysis formula is:

[0096]

[0097] According to the real-time data quality assessment analysis results of each data in each time series data set, a comprehensive data quality assessment is performed on each type of corresponding time series data set. The calculation formula is:

[0098]

[0099] Based on the comprehensive data quality assessment data of each type of time series data set in the corresponding field, the field data value contribution analysis is performed on the current type of time series data. The calculation formula is:

[0100]

[0101] Combined with the quality assessment data and value contribution analysis data corresponding to each type of data in each field, the data in each type of time series data in each field are analyzed for association; the feature coordinates of the data in each type of time series data in each field are constructed respectively, and the feature coordinates are constructed with the field identity feature index of the corresponding data, the type identity feature index, the actual data volume of the corresponding data and the storage time of the data as the dimension data; according to the feature coordinate data of the data in each type of time series data in each field, the data in each type of time series data in each field are traversed and analyzed for association, and the calculation formula is:

[0102]

[0103] By setting the data association threshold R(x), the association between the traversed data is judged by comparing the threshold; if R(m1,m2)<R(x), the two data currently traversed are judged to be weakly associated; if R(m1,m2)≥R(x), the two data currently traversed are judged to be strongly associated; according to the judgment result, by completely traversing the data and performing association analysis, the data with strong association with the target data is annotated and extracted, and according to the association analysis results, the data chain is connected in sequence from strong to weak to construct the associated data chain of the target data;

[0104] The domain type time series data set division of the data stored in the database and the quality assessment and domain contribution analysis data of the corresponding data time series set are recorded and stored; the association analysis results of each data in the database and the associated data chain constructed by the corresponding associated data are fed back and output.

[0105] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A data asset management evaluation method based on data governance, characterized by: The method comprises the following steps: S100, using the authority management port to access the target database, traversing the data stored in the target database by setting up a data retrieval channel, and outputting the traversed data; S200, assigning numbers to the output data of the database traversal based on the associated attributes of the corresponding data, and constructing a time series data set through data integration processing; S300, retrieve the call record data of the corresponding data in each time series data set to perform data call quality analysis on the data in each time series data set; based on the analysis results, perform field contribution rate analysis on the time series data set corresponding to each type of data, and perform association analysis on the data in each field in combination with the analysis data to obtain the associated data chain of the data in each field; The specific steps of S300 are as follows: S301. Based on the corresponding time series data sets of each type in each field, extract the data of each time point in each time series data set respectively; perform call record data query on each extracted data respectively; the call record is the target data stored in the database from the corresponding time point to the current real-time time, the call time and the call number; perform real-time data quality assessment on each target data according to the call data of each target data, and the analysis formula is: Where Dq(f,p,n) is the real-time data quality evaluation value of the data of element number n in the set corresponding to field number f, type number p; α and β are evaluation coefficients; m(f,p,n) and h(f,p,n) are the call times and call time of the data of element number n in the set corresponding to field number f, type number p; T is the current real-time time; t(f,p,n) is the storage time of the data of element number n in the set corresponding to field number f, type number p; According to the real-time data quality assessment analysis results of each data in each time series data set, a comprehensive data quality assessment is performed on each type of corresponding time series data set. The calculation formula is: Among them, Cq(f,p) is the comprehensive type data quality assessment value of the corresponding field number f and the type data time series set number p; based on the comprehensive type data quality assessment data of each type of time series data set in the corresponding field, the field data value contribution analysis is performed on the current type of time series data, and its calculation formula is: Among them, Cv(f,p) is the domain data value contribution value of the time series set of data of type p corresponding to domain f; A(f,p) is the domain contribution impact ratio of data of type p corresponding to domain f; Cq(f,p) max and Cq(f,p) min are the maximum and minimum values ​​of the comprehensive data quality assessment values ​​of each type of data time series set in the corresponding field number f; S302. Combine the quality assessment data and value contribution analysis data corresponding to each type of data in each field, and perform association analysis on the data in each type of time series data in each field; construct feature coordinates for the data in each type of time series data in each field, respectively, and use the field identity feature index, type identity feature index, actual data volume of the corresponding data, and data storage time of the corresponding data as dimension data to construct feature coordinates; perform traversal association analysis on the data in each type of time series data in each field according to the feature coordinate data of the data in each type of time series data in each field, and the calculation formula is: Among them, m1 and m2 are any two data traversed in the corresponding types of time series data in each field; R(m1, m2) is the data association value between data m1 and m2; m1→Cv(f, p) and m1→Cq(f, p) are the domain data value contribution value and the comprehensive type data quality assessment value of the data time series set corresponding to data m1; m2→Cv(f, p) and m2→Cq(f, p) are the domain data value contribution value and the comprehensive type data quality assessment value of the data time series set corresponding to data m2; Z(m1) and Z(m2) are the derived feature vectors of the feature coordinates corresponding to data m1 and m2 respectively; By setting the data association threshold R(x), the association between the traversed data is judged by comparing the threshold; if R(m1,m2)<R(x), the two data currently traversed are judged to be weakly associated; if R(m1,m2)≥R(x), the two data currently traversed are judged to be strongly associated; according to the judgment result, by completely traversing the data and performing association analysis, the data with strong association with the target data is annotated and extracted, and according to the association analysis results, the data chain is connected in sequence from strong to weak to construct the associated data chain of the target data; S400: According to the domain data association analysis result, the association data chain corresponding to the domain data in the database is constructed, stored and output.

2. According to a data asset management evaluation method based on data governance according to claim 1, it is characterized by: The specific steps of S100 using the authority management port to access the target database, traversing the target database storage data by setting up a data retrieval channel, and outputting the traversed data are as follows: S101, access the target database through the access port using the authority administrator, and build a temporary data retrieval channel for the database; S102, traversing the stored data in the database in its entirety, traversing the stored data in the database through a fixed traversal window until the stored data in the database is completely traversed, and outputting the traversed data for analysis.

3. A data asset management evaluation method based on data governance according to claim 2, characterized in that: The specific steps of S200 for assigning numbers to the database traversal output data based on the associated attributes of the corresponding data and constructing a time series data set through data integration processing are as follows: S201, extracting corresponding associated data from the traversed output data in the database; the associated data is the domain data, type data and storage time data of the target data; assigning numbers based on the associated data features of each data, and constructing feature index numbers for each data; S202, based on the characteristic index number of each data, classify and process the data based on the field and type to which the data belongs; Perform time series integration processing on the corresponding types of data in each field after processing, integrate and coordinate the data of the same type in the same field and construct a collection, arrange the data in the collection in time series based on its storage time characteristics, and construct a time series data set of the corresponding type of data in the corresponding field.

4. A data asset management evaluation method based on data governance according to claim 3, characterized in that: The specific steps of constructing, storing and outputting the associated data chain of the corresponding domain data in the database according to the domain data association analysis result in S400 are as follows: S401, recording and storing the domain type time series data set division of the data stored in the database and the quality assessment and domain contribution analysis data of the corresponding data time series set; S402: Feedback and output the associated analysis results of each data in the database and the associated data chain constructed by the corresponding associated data.

5. A data asset management evaluation system based on data governance, applying a data asset management evaluation method based on data governance as described in any one of claims 1 to 4, characterized in that: The system includes a data access module, a data set construction module, a data association evaluation module and a data output module; The data access module uses the authority management port to access the target database, traverses the target database storage data by setting up a data retrieval channel, and outputs the traversed data; the data set construction module assigns a number to the database traversal output data based on the associated attributes of the corresponding data, and constructs a time series data set through data integration processing; The data association evaluation module retrieves the call record data of the corresponding data in each time series data set to perform data call quality analysis on the data in each time series data set; Based on the analysis results, the domain contribution rate analysis is performed on the time series data sets corresponding to each type of data, and the association analysis is performed on the data in each field in combination with the analysis data to obtain the associated data chain of the data in each field; the data output module constructs, stores and outputs the associated data chain of the corresponding field data in the database according to the results of the domain data association analysis.

6. A data asset management and evaluation system based on data governance according to claim 5, characterized in that: The data access module includes a database access unit and a data retrieval unit; The database access unit accesses the target database through the access port of the target database using the identity of the authority administrator, and constructs a temporary data retrieval channel for the database; The data retrieval unit performs a full traversal of the stored data in the database, traverses the stored data in the database through a fixed traversal window, until the stored data in the database is completely traversed, and outputs the traversed data for analysis.

7. A data asset management and evaluation system based on data governance according to claim 6, characterized in that: The data set construction module includes a data processing unit and a time series data set construction unit; The data processing unit extracts the corresponding associated data from the traversed output data in the database; the associated data is the domain data, type data and storage time data of the target data; assigns numbers based on the associated data features of each data, and constructs the feature index number of each data; The time series data set construction unit performs classification processing based on the feature index number of each data and the field and type to which the data belongs; Perform time series integration processing on the corresponding types of data in each field after processing, integrate and coordinate the data of the same type in the same field and construct a collection, arrange the data in the collection in time series based on its storage time characteristics, and construct a time series data set of the corresponding type of data in the corresponding field.

8. A data asset management and evaluation system based on data governance according to claim 7, characterized in that: The data association evaluation module includes a data comprehensive evaluation unit and a data association analysis unit; The data comprehensive evaluation unit extracts data at each time point in each time series data set based on the corresponding types of time series data sets in each field; and performs call record data query on each extracted data; the call record is the target data stored in the database from the corresponding time point to the current real time, the call time and the number of calls; Perform real-time data quality assessment on each target data according to the call data of each target data; According to the real-time data quality assessment and analysis results of each data in each time series data set, a comprehensive data quality assessment is performed on each type of corresponding time series data set; Based on the comprehensive data quality assessment data of various types of time series data sets in the corresponding fields, the domain data value contribution analysis is performed on the current type of time series data; The data association analysis unit combines the quality assessment data and value contribution analysis data corresponding to each type of data in each field, and performs association analysis on the data in the corresponding types of time series data in each field; constructs feature coordinates for the data in the corresponding types of time series data in each field, respectively, and constructs feature coordinates with the field identity feature index of the corresponding data, the type identity feature index, the actual data volume of the corresponding data and the storage time of the data as dimension data; performs traversal association analysis on the data in the corresponding types of time series data in each field according to the feature coordinate data of the data in the corresponding types of time series data in each field; sets a data association threshold, and judges the association between the traversed data by comparing the threshold; and according to the judgment result, constructs an associated data chain of the target data by completely traversing the data and performing association analysis, and by the association analysis result.

9. A data asset management and evaluation system based on data governance according to claim 8, characterized in that: The data output module includes a data recording unit and a data link output unit; The data recording unit records and stores the domain type time series data set division of the data stored in the database and the quality assessment and domain contribution analysis data of the corresponding data time series set; The data chain output unit feeds back and outputs the associated analysis results of each data in the database and the associated data chain constructed by the corresponding associated data.

Images