Master data model design method, system, device and medium based on reverse engineering
Automatically analyze database structure through reverse engineering technology, identify and build master data models, solving the problem of time-consuming and labor-intensive construction of models in traditional methods and the inability to quickly keep up with business changes, and achieving efficient and accurate master data management and flexible data models design.
Patent Information
- Application Number
- CN202411274046.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2025-02-18
- Estimated Expiration
- 2044-09-12
AI Technical Summary
Traditional master data management systems have problems such as time-consuming and labor-intensive construction of models, difficulty in fully capturing the complex correlations and business logic between data, and cannot quickly keep up with the pace of enterprise business changes, resulting in data model lag.
The master data model design method based on reverse engineering is adopted, and the master data model is built by traversing all data tables in the database, collecting structural information, automatically classifying dimension tables and indicator tables, and supporting the verification, optimization and continuous iteration and upgrading of the model.
It significantly improves the efficiency and accuracy of master data management, reduces manual burden, enhances the flexibility and scalability of data models, and provides strong support for enterprises' digital transformation and intelligent decision-making.
Smart Images

Figure CN118779324B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing and information management, and more particularly to a master data model design method, system, device and medium based on reverse engineering. Background Art
[0002] In today's rapidly developing information technology landscape, data has become one of an enterprise's most critical assets, increasingly supporting business decision-making, process optimization, and strategic planning. However, traditional master data management (MDM) systems still face numerous challenges in model building. These systems often rely on cumbersome manual processes, requiring experienced teams of experts to thoroughly analyze business needs and design data models one by one. This process is not only time-consuming and labor-intensive, but also limited by human subjective cognition and understanding. It often fails to fully and accurately capture the complex relationships and business logic between data, resulting in models that fail to fully meet the actual needs of the enterprise.
[0003] Furthermore, as businesses continue to expand and the market environment rapidly changes, data models also require frequent adjustments and optimization. Traditional manual model building methods clearly cannot keep up with this pace of change. As a result, data models often lag behind business practices and fail to provide strong support for real-time decision-making. This lag not only affects operational efficiency but can also cause companies to miss out on market opportunities and face competitive pressure.
[0004] Therefore, how to quickly and accurately build a master data model that meets the actual needs of the enterprise has become an urgent problem to be solved in the current information construction. Summary of the Invention
[0005] In response to the above problems, the purpose of the present invention is to provide a master data model design method, system, device and medium based on reverse engineering, which can automatically perform in-depth analysis of the underlying database, identify and distinguish dimension tables and indicator tables, and then automatically build a master data model based on the structure and content of these tables, and support model verification, optimization and continuous iterative upgrades.
[0006] To achieve the above-mentioned purpose, the present invention is implemented through the following technical solutions:
[0007] In a first aspect, the present invention discloses a master data model design method based on reverse engineering, comprising:
[0008] By traversing all data tables in the database, collecting the structural information of each data table, and classifying the data tables according to the structural information, the data tables are divided into candidate dimension tables and candidate indicator tables;
[0009] Filter dimension tables from candidate dimension tables by analyzing their business attribute description capabilities;
[0010] By analyzing the foreign key associations between the candidate indicator table and the dimension table, the indicator table is selected from the candidate indicator table;
[0011] By extracting key attributes from the dimension table, building a relational model between master data, and mapping the indicator table data to the relational model between master data through foreign key associations, a master data model is generated;
[0012] Verify the master data model using simulated data or actual business data, and optimize the model based on the verification results.
[0013] Furthermore, the structure information of each data table is collected by traversing all data tables in the database, including:
[0014] Connect to different types of database systems through database connectors;
[0015] By traversing all data tables in each database system and using the table structure parser to parse the structural information of each data table;
[0016] The structural information of the data table includes the table name, field name, field type, and whether it is a primary key or a foreign key.
[0017] Furthermore, the data table is classified according to the structural information into candidate dimension tables and candidate index tables, including:
[0018] Based on the structure information, count the percentage of character fields in each data table;
[0019] Determine whether the field ratio exceeds the preset threshold; if so, the data table is a candidate dimension table; if not, the data table is a candidate indicator table.
[0020] Furthermore, the process of screening out dimension tables from the candidate dimension tables by analyzing the business attribute description capabilities of the candidate dimension tables includes:
[0021] For the candidate dimension table, the logical relationship and business meaning between the fields in the table are extracted, and it is determined whether it contains key attribute information describing the business entity; if so, the candidate dimension table meets the definition of a dimension table and is regarded as a dimension table.
[0022] Furthermore, the step of screening out an indicator table from the candidate indicator table by analyzing the foreign key association between the candidate indicator table and the dimension table includes:
[0023] For the candidate indicator table, determine whether there is a foreign key association between the candidate indicator table and the corresponding dimension table; if so, the candidate indicator table is regarded as the indicator table;
[0024] Analyze the aggregation and calculation logic characteristics of the indicator fields in the indicator table and record the analysis results.
[0025] Furthermore, the master data model is generated by extracting key attributes from the dimension table, constructing a relational model between master data, and mapping the indicator table data to the relational model between master data through foreign key associations, including:
[0026] Extract key attributes from dimension tables as master data elements;
[0027] Design and build the relationship model between master data based on the business logic relationship between master data elements;
[0028] Map the data in the indicator table to the relational model between master data through foreign key associations to generate a master data model and form a comprehensive data view that includes business descriptions and quantitative indicators;
[0029] Generate data structure definition, data dictionary and relationship diagram of the master data model.
[0030] Furthermore, the master data model is verified using simulated data or actual business data, and the model is optimized based on the verification results, including:
[0031] Verify the constructed master data model using simulated data or actual business data;
[0032] Adjust and optimize the master data model based on the verification results, and regularly evaluate the effectiveness of the master data model;
[0033] Iterate and upgrade the master data model based on usage results.
[0034] In a second aspect, the present invention further discloses a master data model design system based on reverse engineering, comprising:
[0035] The database scanning and analysis module is used to traverse all data tables in the database, collect the structural information of each data table, classify the data tables according to the structural information, and divide the data tables into candidate dimension tables and candidate indicator tables;
[0036] The module for fine-grained partitioning of dimension tables and indicator tables is used to select dimension tables from candidate dimension tables by analyzing their business attribute description capabilities; and to select indicator tables from candidate indicator tables by analyzing foreign key associations between candidate indicator tables and dimension tables.
[0037] The master data model construction module is used to extract key attributes from dimension tables, construct a relational model between master data, and map the indicator table data to the relational model between master data through foreign key associations to generate a master data model;
[0038] The model verification and optimization module is used to verify the master data model through simulated data or actual business data, and optimize the model based on the verification results.
[0039] In a third aspect, the present invention further discloses a master data model design device based on reverse engineering, comprising:
[0040] A memory for storing a master data model design program based on reverse engineering;
[0041] A processor is used to implement the steps of the master data model design method based on reverse engineering as described in any one of the above items when executing the master data model design program based on reverse engineering.
[0042] In a fourth aspect, the present invention also discloses a readable storage medium, on which a master data model design program based on reverse engineering is stored. When the master data model design program based on reverse engineering is executed by a processor, the steps of the master data model design method based on reverse engineering as described in any one of the above items are implemented.
[0043] Compared with the existing technology, the beneficial effect of the present invention is that: the present invention discloses a master data model design method, system, device and medium based on reverse engineering, which uses reverse engineering technology to conduct in-depth analysis of the underlying database, identify and distinguish dimension tables and indicator tables, and then automatically construct a master data model according to the structure and content of these tables to realize the design, reuse and management of the master data model.
[0044] This invention significantly improves the efficiency and accuracy of master data management by automatically analyzing database structures and intelligently identifying and constructing master data models. This not only reduces the manual burden but also enhances the flexibility and scalability of data models, providing strong support for digital transformation and intelligent decision-making in enterprises.
[0045] It can be seen that compared with the prior art, the present invention has outstanding substantial features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0047] Figure 1 It is a method flow chart of a specific embodiment of the present invention.
[0048] Figure 2It is a system structure diagram of a specific implementation method of the present invention.
[0049] In the figure, 1. Database scanning and analysis module; 2. Dimension table and indicator table fine division module; 3. Master data model construction module; 4. Model verification and optimization module. DETAILED DESCRIPTION
[0050] In order to enable those skilled in the art to better understand the present invention, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0051] See also Figure 1 As shown, this embodiment provides a master data model design method based on reverse engineering, including the following steps:
[0052] S1: By traversing all data tables in the database, collecting the structural information of each data table, classifying the data tables according to the structural information, and dividing the data tables into candidate dimension tables and candidate indicator tables.
[0053] In a specific embodiment, first, different types of database systems are connected through database connectors.
[0054] We use automated tools to traverse all data tables in each database system and use a table structure analyzer to parse and collect the structural information of each data table. The structural information of the data table includes the table name, field name, field type, whether it is a primary key or a foreign key, etc.
[0055] Then, based on the structural information, the proportion of character fields in each data table is counted to determine whether the field proportion exceeds the preset threshold. Based on the judgment result, the data table is preliminarily classified as a candidate dimension table or a candidate indicator table.
[0056] Specifically, if the proportion of character fields in a table exceeds 70% and it contains a large number of descriptive fields, it is considered a candidate dimension table; conversely, if the proportion of character fields does not exceed 70%, it means that the data table has a high proportion of numeric fields and is often associated with candidate dimension tables through foreign keys, and it is considered a candidate indicator table.
[0057] Among them, the above-mentioned preliminary classification table based on features such as field type can be implemented using a preset preliminary classifier.
[0058] S2: Filter out dimension tables from the candidate dimension tables by analyzing their business attribute description capabilities.
[0059] In a specific implementation, for a candidate dimension table, the logical relationship and business meaning between fields in the table are extracted, and it is determined whether it contains key attribute information describing the business entity; if so, the candidate dimension table meets the definition of a dimension table and is regarded as a dimension table.
[0060] The purpose of this step is to further analyze the logical relationships and business meanings between fields in the candidate dimension table to confirm whether it truly meets the definition of a dimension table (that is, it contains attribute information used to describe business entities).
[0061] S3: Filter out the indicator table from the candidate indicator table by analyzing the foreign key association between the candidate indicator table and the dimension table.
[0062] In a specific implementation, for a candidate indicator table, a determination is first made as to whether a foreign key relationship exists between the candidate indicator table and the corresponding dimension table. If so, the candidate indicator table is considered the indicator table. Furthermore, in this step, the aggregation and computational logic properties of the indicator fields in the indicator table may be analyzed, and the analysis results recorded.
[0063] For example, for candidate indicator tables, verify their foreign key associations with dimension tables to ensure they accurately reflect the quantitative results of business activities (e.g., large amounts of transaction flow data and indicator data). Also, analyze the aggregation and calculation logic of indicator fields to prepare for subsequent data model optimization.
[0064] In addition, it should be noted that the above steps S2 and S3 implement a fine division of the dimension table and the indicator table. To implement the corresponding method, a logic analyzer can be used to analyze the logical relationship between fields, field data types, and business meanings, a foreign key validator can be used to check foreign key associations between tables, and a fine classifier can be used to perform fine classification based on the in-depth analysis results.
[0065] S4: By extracting key attributes from the dimension table, a relational model between master data is constructed, and the indicator table data is mapped to the relational model between master data through foreign key associations to generate a master data model.
[0066] In a specific implementation, key attributes are first extracted from dimension tables as master data elements. Then, based on the business logic relationships between master data elements, a relational model between master data is designed and constructed. Finally, data from the indicator tables is mapped to the relational model between master data through foreign key associations, generating a master data model and forming a comprehensive data view that includes business descriptions and quantitative indicators. Furthermore, a data structure definition, data dictionary, and relationship diagram for the master data model are generated.
[0067] As an example, this step specifically includes the following sub-steps:
[0068] S401: Based on the finely divided dimension tables, use the master data element extractor to extract key attributes from the dimension tables as master data elements and build the master data model framework. Considering the quality requirements of the master data, such as uniqueness, completeness, and accuracy, design corresponding data verification rules and business processes.
[0069] S402: Based on the business logic relationships between master data elements, use the relationship model builder to build a relationship model between master data, such as parent-child relationships, association relationships, etc., to ensure that the master data model can fully reflect the complex structure of the enterprise business entities.
[0070] S403: Use the data mapper to map the data in the indicator table to the master data model through foreign key associations, forming a comprehensive data view that includes business descriptions and quantitative indicators, providing strong support for data analysis, report generation, etc. (In the scenario of the method objective of master data management, this step S403 is optional).
[0071] In addition, you can also use the model generator to generate relevant documents and charts of the master data model according to your needs.
[0072] S5: Verify the master data model using simulated data or actual business data, and optimize the model based on the verification results.
[0073] In a specific implementation, the constructed master data model is first verified using simulated data or actual business data. The master data model is then adjusted and optimized based on the verification results, and its effectiveness is regularly evaluated. Furthermore, the master data model can be iteratively upgraded based on the effectiveness of its use.
[0074] For example, the constructed master data model is first verified using simulated data or actual business data to check data consistency, integrity, and model performance. A data validation engine can be used to perform data consistency and integrity checks, while a performance evaluator can be used to evaluate model performance.
[0075] Then, based on the validation results, make necessary adjustments and optimizations to the model, including adjusting field definitions, optimizing data association logic, and enhancing data validation rules. Manual review can be performed as necessary. The Adjustment Optimizer can automatically or assistedly adjust and optimize the model based on the validation results, while the Validation Report Generator can generate a model validation report and optimization recommendations.
[0076] In addition, the effectiveness of the master data model is regularly evaluated, user feedback is collected, and continuous iteration and upgrades are carried out to ensure that the model always adapts to the needs of corporate business development.
[0077] As an example, based on a specific data context, this embodiment further discloses a master data model design method based on reverse engineering, including the following steps:
[0078] 1. Database scanning and analysis:
[0079] First, use the database connector to configure the database connection parameters and successfully connect to the e-commerce platform's MySQL database. The e-commerce platform has a large database containing product information, customer information, order information, and other data.
[0080] Then, use the table structure parser to traverse all tables in the database and parse out the name, field name, field type and other information of each table.
[0081] At this time, the preliminary classifier is used to analyze the field types and preliminarily classify the basic product information table (containing many character fields such as product ID, product name, product description, etc.) as a candidate dimension table; the sales data table (containing many numerical fields such as product ID, sales volume, sales revenue, and containing product ID as a foreign key) as a candidate indicator table.
[0082] Finally, a preliminary classification report is output, in which all candidate dimension tables and candidate indicator tables are listed.
[0083] 2. Fine division of dimension tables and indicator tables:
[0084] First, use a logic analyzer to thoroughly analyze the product basic information table to confirm that its fields, such as product name, product category, and brand, are descriptive and conform to the dimension table definition. Then, use a foreign key validator to verify that the product ID field in the sales data table is indeed linked to the product basic information table as a foreign key.
[0085] Based on the above analysis, the refined classifier confirms that the product basic information table is the official dimension table, and the sales data table is the official indicator table. Finally, a refined classification report is output, which details the field descriptions and relationships of the dimension and indicator tables.
[0086] 3. Master data model construction:
[0087] First, use the Master Data Element Extractor to extract product IDs, product names, product categories, and brands from the product basic information table as master data elements. Next, use the Relational Model Builder to design the relationships between products and customers (assuming that every time a customer purchases a product, there's a purchase record table with customer and product IDs as foreign keys). At this point, use the Data Mapper to map the data in the sales data table to the product master data model using product IDs, creating a comprehensive data view encompassing product sales. Simultaneously, use the Model Builder to generate the product master data model's data structure definition (e.g., XML Schema or JSON Schema), a data dictionary (describing the meaning and constraints of each field), and a relationship diagram (showing the relationships between products and customers).
[0088] Finally, output relevant documents and diagrams of the product master data model.
[0089] 4. Model validation and optimization:
[0090] Use a data validation engine to verify the product master data model's data consistency (e.g., uniqueness of product IDs), completeness (e.g., all products must have names and categories), and performance (e.g., query response time) using simulated or actual business data. Simultaneously, use a performance evaluator to assess the model's performance under large data volumes and identify potential bottlenecks.
[0091] After verification is complete, use the optimization tool to make necessary adjustments to the model based on the results, such as optimizing query statements and adding indexes. At the same time, use the verification report generator to generate a model verification report summarizing the verification results and optimization suggestions.
[0092] Finally, the model verification report is output, including data consistency, integrity verification results and performance evaluation report, as well as optimization suggestions.
[0093] Through the above implementation process, e-commerce platforms can automatically build and optimize the master data models of products and customers based on reverse engineering methods, improve the efficiency and accuracy of data management, and provide support for subsequent business analysis and decision-making.
[0094] See also Figure 2 As shown, the present invention also discloses a master data model design system based on reverse engineering, including: a database scanning and analysis module 1, a dimension table and indicator table fine division module 2, a master data model construction module 3 and a model verification and optimization module 4.
[0095] The database scanning and analysis module 1 is used to traverse all data tables in the database, collect structural information of each data table, classify the data tables according to the structural information, and divide the data tables into candidate dimension tables and candidate index tables.
[0096] The dimension table and indicator table fine division module 2 is used to screen out dimension tables from candidate dimension tables by analyzing the business attribute description capabilities of candidate dimension tables; and to screen out indicator tables from candidate indicator tables by analyzing the foreign key associations between candidate indicator tables and dimension tables.
[0097] The master data model construction module 3 is used to construct a relational model between master data by extracting key attributes from the dimension table, and map the indicator table data to the relational model between master data through foreign key association to generate a master data model.
[0098] The model verification and optimization module 4 is used to verify the master data model through simulation data or actual business data, and optimize the model according to the verification results.
[0099] The specific implementation of the master data model design system based on reverse engineering in this embodiment is basically the same as the specific implementation of the master data model design method based on reverse engineering described above, and will not be repeated here.
[0100] The present invention also discloses a master data model design device based on reverse engineering, comprising a processor and a memory; wherein, when the processor executes the master data model design program based on reverse engineering stored in the memory, the steps of the master data model design method based on reverse engineering as described in any one of the above items are implemented.
[0101] Furthermore, the master data model design apparatus based on reverse engineering in this embodiment may further include:
[0102] The input interface is used to obtain a reverse engineering-based master data model design program imported from an external device and save the obtained reverse engineering-based master data model design program to the memory. It can also be used to obtain various instructions and parameters transmitted by an external terminal device and transmit them to the processor so that the processor can use these various instructions and parameters to perform corresponding processing. In this embodiment, the input interface can specifically include but is not limited to a USB interface, a serial interface, a voice input interface, a fingerprint input interface, a hard disk read interface, etc.
[0103] The output interface is used to output various data generated by the processor to the terminal device connected to it, so that other terminal devices connected to the output interface can obtain various data generated by the processor. In this embodiment, the output interface can specifically include but is not limited to a USB interface, a serial interface, etc.
[0104] A communication unit is configured to establish a remote communication connection between the reverse engineering-based master data model design apparatus and an external server, so that the reverse engineering-based master data model design apparatus can mount the image file to the external server. In this embodiment, the communication unit may specifically include, but is not limited to, a remote communication unit based on wireless communication technology or wired communication technology.
[0105] The keyboard is used to obtain various parameter data or instructions input by the user by tapping the keycaps in real time.
[0106] A display is used to display in real time relevant information of a design process based on a master data model of reverse engineering.
[0107] The mouse can be used to assist users in inputting data and simplify user operations.
[0108] The present invention also discloses a readable storage medium, wherein the readable storage medium includes a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable hard disk, a CD-ROM, or any other form of storage medium known in the technical field. The readable storage medium stores a master data model design program based on reverse engineering, and when the master data model design program based on reverse engineering is executed by a processor, the steps of the master data model design method based on reverse engineering as described in any of the above items are implemented.
[0109] In summary, the present invention can automatically conduct in-depth analysis of the underlying database, identify and distinguish dimension tables from indicator tables, and then automatically build a master data model based on the structure and content of these tables, and support model verification, optimization and continuous iterative upgrades.
[0110] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. References to the same or similar parts between the various embodiments are sufficient. The methods disclosed in the embodiments are described briefly because they correspond to the systems disclosed in the embodiments. For relevant details, refer to the method description.
[0111] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.
[0112] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.
[0113] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0114] In addition, the functional modules in the various embodiments of the present invention may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit.
[0115] Similarly, each processing unit in each embodiment of the present invention may be integrated into one functional module, or each processing unit may exist physically, or two or more processing units may be integrated into one functional module.
[0116] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0117] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.
[0118] The above is a detailed introduction to the master data model design method, system, device and readable storage medium based on reverse engineering provided by the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified in several ways, and these improvements and modifications also fall within the scope of protection of the present invention.
Claims
1. A master data model design method based on reverse engineering, characterized in that: include: By traversing all data tables in the database, collecting the structural information of each data table, classifying the data tables according to the structural information, and dividing the data tables into candidate dimension tables and candidate indicator tables; Filter out dimension tables from candidate dimension tables by analyzing their business attribute description capabilities; By analyzing the foreign key association between the candidate indicator table and the dimension table, the indicator table is selected from the candidate indicator table; By extracting key attributes from the dimension table, building a relational model between master data, and mapping the indicator table data to the relational model between master data through foreign key associations, a master data model is generated; Verify the master data model using simulated data or actual business data, and optimize the model based on the verification results; The method traverses all data tables in the database and collects the structural information of each data table, including: Connect different types of database systems through database connectors; By traversing all data tables in each database system and using the table structure parser to parse the structural information of each data table; The structural information of the data table includes the table name, field name, field type, and whether it is a primary key or a foreign key; The data table is classified according to the structural information to be a candidate dimension table and a candidate index table, including: Based on the structure information, count the percentage of character fields in each data table; Determine whether the field ratio exceeds the preset threshold; if so, the data table is a candidate dimension table; if not, the data table is a candidate indicator table; The step of analyzing the business attribute description capability of the candidate dimension tables and selecting the dimension tables from the candidate dimension tables includes: For the candidate dimension table, the logical relationship and business meaning between the fields in the table are extracted, and it is determined whether it contains key attribute information describing the business entity; if so, the candidate dimension table meets the definition of a dimension table and is regarded as a dimension table.
2. The master data model design method based on reverse engineering according to claim 1, characterized in that: The step of analyzing the foreign key association between the candidate indicator table and the dimension table to select the indicator table from the candidate indicator table includes: For the candidate indicator table, determine whether there is a foreign key association between the candidate indicator table and the corresponding dimension table; if so, the candidate indicator table is regarded as the indicator table; Analyze the aggregation and calculation logic characteristics of the indicator fields in the indicator table, and record the analysis results.
3. The master data model design method based on reverse engineering according to claim 2, characterized in that: The method extracts key attributes from the dimension table, constructs a relational model between master data, and maps the indicator table data to the relational model between master data through foreign key association to generate a master data model, including: Extract key attributes from dimension tables as master data elements; Design and build the relationship model between master data according to the business logic relationship between master data elements; Map the data in the indicator table to the relational model between master data through foreign key association, generate the master data model, and form a comprehensive data view including business description and quantitative indicators; Generate data structure definition, data dictionary and relationship diagram for the master data model.
4. The master data model design method based on reverse engineering according to claim 3 is characterized in that: The master data model is verified by simulation data or actual business data, and the model is optimized according to the verification results, including: Verify the constructed master data model through simulation data or actual business data; Adjust and optimize the master data model based on the verification results, and regularly evaluate the effectiveness of the master data model; Iterate and upgrade the master data model based on usage results.
5. A master data model design system based on reverse engineering, characterized in that: The system adopts the master data model design method based on reverse engineering as described in any one of claims 1 to 4; The system comprises: The database scanning and analysis module is used to traverse all data tables in the database, collect the structural information of each data table, classify the data tables according to the structural information, and divide the data tables into candidate dimension tables and candidate indicator tables; The module for finely dividing dimension tables and indicator tables is used to filter out dimension tables from candidate dimension tables by analyzing the business attribute description capabilities of candidate dimension tables; and to filter out indicator tables from candidate indicator tables by analyzing the foreign key associations between candidate indicator tables and dimension tables; The master data model building module is used to extract key attributes from the dimension table, build the relationship model between the master data, and map the indicator table data to the relationship model between the master data through foreign key association to generate the master data model; The model verification and optimization module is used to verify the master data model through simulation data or actual business data, and optimize the model based on the verification results.
6. A master data model design device based on reverse engineering, characterized in that: include: A memory for storing a master data model design program based on reverse engineering; A processor is used to implement the steps of the master data model design method based on reverse engineering as described in any one of claims 1 to 4 when executing the master data model design program based on reverse engineering.
7. A readable storage medium, characterized in that: The readable storage medium stores a master data model design program based on reverse engineering, and when the master data model design program based on reverse engineering is executed by a processor, the steps of the master data model design method based on reverse engineering as described in any one of claims 1 to 4 are implemented.
Citation Information
Patent Citations
Method and system for creating master data model
CN116185360A
Automated master data classification and curation using machine learning
US20210209159A1