Data management method
By using deep learning binary classification models and BERT-BiLSTM models, the problems of data quality assurance, lack of objectivity in evaluation standards, and fusion of multi-source heterogeneous data in data governance are solved, achieving high-precision prediction and closed-loop optimization, and improving the efficiency of data governance.
Patent Information
- Application Number
- CN202511176473.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-21
- Publication Date
- 2025-11-21
AI Technical Summary
Traditional data governance methods suffer from problems such as difficulty in ensuring data quality, lack of objectivity and comparability in evaluation standards, insufficient integration of multi-source heterogeneous data, limited accuracy of prediction models, and lack of closed-loop optimization in governance processes.
We employ a deep learning-based binary classification model for data cleaning and correction, construct comparable quantitative indicators across datasets, integrate heterogeneous data from multiple sources, use the BERT-BiLSTM model for high-precision prediction, and form a closed-loop optimization mechanism.
It enables automated improvement of data quality, objective quantitative assessment, support for intelligent decision-making, and promotes continuous self-optimization of governance processes.
Smart Images
Figure CN120994652A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data governance technology, specifically a data governance method. Background Technology
[0002] In today's data-driven decision-making environment, efficient data governance is crucial for realizing data value. However, traditional data governance methods face significant challenges: Data quality assurance is difficult: data cleaning methods relying on manual rules or simple statistics are inefficient, failing to effectively identify and correct incomplete, erroneous, and subjective biases, leading to insufficient data reliability; Evaluation standards lack objectivity and comparability: especially in specialized fields such as disease management, subjective scoring methods struggle to quantify severity, and evaluation results cannot be compared horizontally across datasets of different sizes or sources; Insufficient fusion of multi-source heterogeneous data: static and dynamic data, such as classification attributes and historical states, are stored in isolation, forming information silos and hindering comprehensive analysis of the entire entity lifecycle; Predictive model accuracy is limited: single algorithms struggle to simultaneously capture global semantic features and local temporal dependencies; Governance processes lack closed-loop optimization: existing solutions are mostly unidirectional pipelines, unable to dynamically optimize data collection standards based on analysis results, resulting in difficulty in continuously improving governance effectiveness. Therefore, a data governance method that can systematically address data quality, objective quantification, multi-source fusion, intelligent prediction, and closed-loop optimization is urgently needed. Summary of the Invention
[0003] Therefore, in order to address the above-mentioned shortcomings, the present invention provides a data governance method.
[0004] This invention is implemented as follows: a data governance method is constructed, comprising a data governance system including a validity judgment module with a built-in deep learning-based binary classification model to learn from datasets labeled with validity; a quantitative index construction module connected to the output of the validity judgment module, which generates cross-dataset comparable severity quantitative indicators based on the number of disease categories and preset scaling values; a multi-source data integration module that integrates heterogeneous data of classification attributes and historical states; storing static standard data and dynamic peak data by dataset unit; an intelligent prediction module including a BERT submodule and a BiLSTM submodule; and a closed-loop optimization module that updates the data collection specifications in reverse based on the output of the intelligent prediction module.
[0005] A data governance approach, comprising the following steps:
[0006] Step 1, data cleaning and correction, is used to address incomplete, erroneous, and subjective biases in the data; a validity judgment module is adopted, based on a deep learning-based binary classification model design method, to learn the dataset after validity labeling;
[0007] Formula 1 is used: Calculate the data accuracy P, where N TP N represents the number of samples that are actually positive and are predicted to be positive. FP This represents the number of samples that were actually negative but were predicted as positive.
[0008] Formula 2 is used: Calculate the recall rate R, where N FN This represents the number of samples that were actually positive but were predicted as negative.
[0009] Formula 3 is used: Calculate the harmonic mean F of the recall rate; where the harmonic mean F indicates the validity of the data and is used for filtering invalid data;
[0010] Step 2: Feature engineering and quantitative indicator construction to replace subjective scoring and achieve objective, quantifiable evaluation, according to Formula 4: Calculate the severity index S of disease data, where C is the number of data classes in the dataset, and n i Let l be the number of disease data of type i in this dataset. i The disease scale is defined for the i-th type of disease data to achieve comparability of indicators for datasets of different sizes.
[0011] Step 3: Multi-source data integration, used to fuse heterogeneous data such as classification and historical status. The database stores data in units of datasets, specifically static standard data and dynamic peak data.
[0012] Step 4: Intelligent modeling and prediction, used to build an automated, high-precision prediction model; employing a BERT-BiLSTM fusion model: BERT: extracts global semantics; BiLSTM: captures local temporal features;
[0013] Step 5: Application and Iterative Optimization; used to support decision-making and optimize data quality in a closed loop, and the prediction results are used to optimize data collection specifications in reverse.
[0014] The present invention has the following advantages: The present invention provides a data governance method through improvements, which, compared with similar devices, have the following improvements:
[0015] The data governance method described in this invention automatically cleans and filters invalid data using a deep learning binary classification model, constructs objective quantitative indicators comparable across datasets to replace subjective ratings, integrates multi-source heterogeneous data, and uniformly stores static standards and dynamic peak data according to datasets; employs a BERT-BiLSTM fusion model for high-precision prediction; and forms a closed-loop mechanism to use prediction results to optimize data collection specifications, thereby systematically improving data quality, achieving objective assessment, supporting intelligent decision-making, and promoting continuous self-optimization of the governance process. Attached Figure Description
[0016] Figure 1 This is a schematic diagram of the process of this invention. Detailed Implementation
[0017] The following is in conjunction with the appendix Figure 1 The principles and features of the present invention are described below. The examples given are for illustrative purposes only and are not intended to limit the scope of the invention. The invention is described more specifically in the following paragraphs by way of example with reference to the accompanying drawings. It should be noted that the drawings are in a very simplified form and use non-precise proportions, and are only used to facilitate and clarify the illustration of the embodiments of the invention.
[0018] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.
[0019] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "setting" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. The embodiments of this invention will now be described according to its overall structure.
[0020] Please see Figure 1 This invention discloses a data governance method, comprising a data governance system, specifically a validity judgment module with a built-in deep learning-based binary classification model for learning from datasets labeled with validity; a quantitative index construction module connected to the output of the validity judgment module, which generates cross-dataset comparable severity quantitative indicators based on the number of disease categories and preset scaling values; a multi-source data integration module that integrates heterogeneous data of classification attributes and historical states; and stores static standard data and dynamic peak data by dataset unit; an intelligent prediction module including a BERT submodule and a BiLSTM submodule; and a closed-loop optimization module that updates the data collection specifications in reverse based on the output of the intelligent prediction module.
[0021] A data governance approach includes the following steps:
[0022] Step 1, data cleaning and correction, is used to address incomplete, erroneous, and subjective biases in the data; a validity judgment module is adopted, based on a deep learning-based binary classification model design method, to learn the dataset after validity labeling;
[0023] Formula 1 is used: Calculate the data accuracy P, where N TP N represents the number of samples that are actually positive and are predicted to be positive. FP This represents the number of samples that were actually negative but were predicted as positive.
[0024] Formula 2 is used: Calculate the recall rate R, where N FN This represents the number of samples that were actually positive but were predicted as negative.
[0025] Formula 3 is used: Calculate the harmonic mean F of the recall rate; where the harmonic mean F indicates the validity of the data and is used for filtering invalid data;
[0026] Step 2: Feature engineering and quantitative indicator construction to replace subjective scoring and achieve objective, quantifiable evaluation, according to Formula 4: Calculate the severity index S of disease data, where C is the number of data classes in the dataset, and n i Let l be the number of disease data of type i in this dataset. i The disease scale is defined for the i-th type of disease data to achieve comparability of indicators for datasets of different sizes.
[0027] Step 3: Multi-source data integration, used to fuse heterogeneous data such as classification and historical status. The database stores data in units of datasets, specifically static standard data and dynamic peak data.
[0028] Step 4: Intelligent modeling and prediction, used to build an automated, high-precision prediction model; employing a BERT-BiLSTM fusion model: BERT: extracts global semantics; BiLSTM: captures local temporal features;
[0029] Step 5: Application and Iterative Optimization; used to support decision-making and optimize data quality in a closed loop, and the prediction results are used to optimize data collection specifications in reverse.
[0030] This invention provides an improved data governance method that automatically cleans and filters invalid data using a deep learning binary classification model, constructs objective quantitative indicators comparable across datasets to replace subjective ratings, integrates multi-source heterogeneous data, and uniformly stores static standards and dynamic peak data according to datasets; employs a BERT-BiLSTM fusion model for high-precision prediction; and forms a closed-loop mechanism to use prediction results to optimize data collection specifications, thereby systematically improving data quality, achieving objective assessment, supporting intelligent decision-making, and promoting continuous self-optimization of the governance process.
[0031] The above description shows and illustrates the basic principles, main features, and advantages of the present invention. Standard parts used in the present invention can be purchased from the market, and irregular parts can be customized according to the description and drawings. The specific connection methods of each part adopt conventional methods such as bolts, rivets, and welding that are mature in the prior art. The machinery, parts, and equipment adopt conventional models in the prior art, and the circuit connection adopts conventional connection methods in the prior art, which will not be described in detail here.
[0032] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A data governance method, comprising the following steps: Step 1: Data cleaning and correction, used to resolve data incompleteness, errors, and subjective biases; A deep learning-based binary classification model design method is adopted for the validity judgment module to learn the dataset after validity labeling; Formula 1 is used: Calculate the data accuracy P, where N TP N represents the number of samples that are actually positive and are predicted to be positive. FP This represents the number of samples that were actually negative but were predicted as positive. Formula 2 is used: Calculate the recall rate R, where N FN This represents the number of samples that were actually positive but were predicted as negative. Formula 3 is used: Calculate the harmonic mean F of the recall rate; where the harmonic mean F indicates the validity of the data and is used for filtering invalid data.
2. The data governance method according to claim 1, characterized in that: Following step one is step two: feature engineering and quantitative indicator construction, used to replace subjective scoring and achieve objective, quantifiable evaluation, according to formula four: Calculate the severity index S of disease data, where C is the number of data classes in the dataset, and n i Let l be the number of disease data of type i in this dataset. i Let be the disease scale corresponding to the i-th type of disease data, so as to achieve comparability of indicators for datasets of different sizes.
3. The data governance method according to claim 2, characterized in that: Step two is followed by step three, which involves multi-source data integration to fuse heterogeneous data such as classification and historical status. The database stores data in units of datasets, specifically static standard data and dynamic peak data.
4. The data governance method according to claim 3, characterized in that: Step three is followed by step four, intelligent modeling and prediction, which is used to establish an automated and high-precision prediction model; a BERT-BiLSTM fusion model is adopted: BERT: extracts global semantics; BiLSTM: captures local temporal features.
5. The data governance method according to claim 4, characterized in that: Step four is followed by step five, application and iterative optimization; used to support decision-making and optimize data quality in a closed loop, and the prediction results are used to optimize the data collection specifications in reverse.
6. The data governance method according to claim 5, characterized in that: It also includes a data governance system, specifically comprising: a validity assessment module, which has a built-in deep learning-based binary classification model to learn from datasets labeled with validity; a quantitative indicator construction module, connected to the output of the validity assessment module, which generates cross-dataset comparable severity quantitative indicators based on the number of disease categories and preset scaling values; a multi-source data integration module, which integrates heterogeneous data of classification attributes and historical states; and stores static standard data and dynamic peak data by dataset unit; an intelligent prediction module, including BERT sub-modules and BiLSTM sub-modules; and a closed-loop optimization module, which updates the data collection specifications in reverse based on the output of the intelligent prediction module.