Master data management method and system based on enterprise data governance

Abstract

The invention relates to the technical field of data processing, in particular to a main data management method and system based on enterprise data governance, and the method comprises the steps: defining a basic model structure based on an enterprise general main data object, and storing the basic model structure in a metadata form; obtaining a business system change event and a manual expansion demand and converting the event and the demand into a structured description; zero-coding extension configuration of attributes, classification and association relationships is completed through a visual interface; executing interface adaptation, data mapping adaptation and rule conflict detection; and synchronizing the effective extension model to a service layer, a cleaning layer and a monitoring layer. The system comprises a demand awareness module, a dynamic modeling module, an extension configuration module, an adaptation engine module and a life cycle management module. According to the method, full-process management of the master data model from demand capture to business landing is realized, and the model expansion efficiency and the business adaptation capability are improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a master data management method and system based on enterprise data governance. Background Art

[0002] Master data in enterprise data governance is a collection of fundamental data entities with high business value, shared across business areas. These typically cover core business objects such as customers, suppliers, products, and employees. Master data is characterized by long-term relative stability, clear business attributes, and repeated references across multiple systems or business processes. As a key component of data governance, master data management focuses on establishing an authoritative, enterprise-wide source of master data through standardized definitions, unified maintenance, and full lifecycle management. This provides a consistent, accurate, and complete data benchmark for cross-departmental business collaboration, data analysis applications, and system integration.

[0003] Master data management approaches based on enterprise data governance face the following pain points in terms of business adaptability: existing approaches lack the ability to expand and adjust data models in response to dynamic changes in enterprise business, making it difficult to quickly respond to the demands of new business scenarios. For example, when an enterprise adds a new product line due to market expansion, existing master data management requires manual redefinition of model elements such as master data characteristics, classifications, attributes, and unique identifiers. This is time-consuming and prone to human error, leading to delayed master data updates, impacting the timeliness of cross-system data sharing and the collaborative efficiency of business processes, and limiting the data's ability to support business innovation. Summary of the Invention

[0004] In response to the shortcomings of existing technologies, the present invention provides a master data management method and system based on enterprise data governance to solve the problem of insufficient data model expansion and adjustment capabilities in master data management, making it difficult to quickly respond to the dynamic changes in enterprise business needs.

[0005] In order to solve the above technical problems, the specific technical solutions of the present invention are as follows: In a first aspect, the present invention provides a master data management method based on enterprise data governance, comprising: Step 1: Define the basic model structure based on the enterprise's common master data objects. The basic model structure includes core attributes, classification system, unique identification rules, and association relationships. It is stored in the form of metadata in a model repository that supports version management and serves as a benchmark for subsequent expansion. Step 2: Obtain business system change events and manually submitted extension requirements, and convert the events and requirements into structured extension requirements that include event types, target master data models, and extension detail fields. Step 3: Based on the structured extension requirements and the basic model metadata stored in the model warehouse, complete the attribute, classification, and relationship extension configuration of the master data model through a visual interface to generate extended model metadata; Step 4: Perform interface adaptation, data mapping adaptation, and rule conflict detection on the extended model metadata. After verification, mark it as valid. Step 5: Synchronize the effective extended model metadata to the service layer, cleaning layer, and monitoring layer. The service layer is used for master data service interface management, the cleaning layer is used for data cleaning processing, and the monitoring layer is used for data quality monitoring. This is used for extended synchronization and monitoring throughout the entire life cycle of master data.

[0006] Furthermore, the master data management method based on enterprise data governance of the present invention obtains business system change events by deploying an event monitoring component for log collection in the database change log and API call log of the business system, matching business change keywords through regular expressions, and extracting event context including change time, involved systems, and affected business objects; Obtaining manually submitted extension requirements includes receiving the extension object, new attribute name, and category adjustment range filled in by business personnel through the management console web form interface open to business personnel; The event context extracted by the event listening component and the manually submitted extension requirements are uniformly converted into structured extension requirements for the event type, target master data model, and extension detail fields in step 2 through the message queue.

[0007] Furthermore, the master data management method based on enterprise data governance of the present invention has a basic model structure that is a model framework based on the definition of enterprise common master data objects, specifically including core attributes, classification system, unique identification rules and association relationships; The basic model structure is stored in the form of a metadata descriptor in JSON format. The metadata descriptor includes the model version number, the parent model reference, the type field that maps the basic model structure elements, and the constraint condition field corresponding to the unique identification rules and association relationships; The model warehouse stores basic model metadata including historical versions through database transactions.

[0008] Furthermore, the master data management method based on enterprise data governance of the present invention further includes: reading the defined basic model metadata from the model warehouse, and based on the real-time attribute list, classification system and association relationship of the basic model, including: Add attributes to the attribute list area by dragging and dropping, enter the attribute name, data type, and constraint rules, associate the enterprise terminology base, and generate attribute extended metadata fragments; In the classification system area, the real-time classification system of the basic model is displayed in a tree diagram. Users can right-click and select nodes to add sub-categories, record classification versions and mapping rules between old and new classifications, and generate classification extension metadata fragments. In the relationship area, use the relationship diagram component to display the existing relationships of the basic model, select new relationship objects, configure the relationship rules, and generate the relationship metadata fragment; After the extension is completed, the extended metadata fragments of attributes, classifications, and associations are merged with the basic model metadata to generate extended model metadata marked as pending verification.

[0009] Furthermore, the master data management method based on enterprise data governance of the present invention further includes: Call the business system API gateway metadata service to obtain the field definition of the real-time master data interface, parse the newly added attributes in the extended model metadata, automatically generate the interface field extension script, and update the OpenAPI document of the master data service interface; Data lineage analysis tools display the topological relationship of master data; Read the data quality rule library, verify whether the newly added attributes of the extended model metadata conflict with existing rules, generate a report including the conflict rule ID and the cause of the conflict, and push it to the administrator console for review; If no conflict is detected or the conflict has been resolved, the extended model metadata is marked as valid.

[0010] Furthermore, the master data management method based on enterprise data governance of the present invention further includes: publishing the expanded master data service interface field definition via a message bus, notifying the business system to subscribe to the topic, and triggering the business system local interface adaptation; The data cleaning engine reads the attribute constraint rules of the effective extended model metadata from the model warehouse, writes the rules into the cleaning task configuration, and automatically applies the rules to newly connected or updated master data instances; Extract the extended attributes of the effective extended model metadata from the model warehouse, add new records to the monitoring indicator configuration table, collect field data of the master data instance through scheduled tasks, calculate the missing rate and display it on the dashboard; after the synchronization of each layer is completed, the system records the final effective status of the extended model.

[0011] Furthermore, in the master data management method based on enterprise data governance of the present invention, the business systems include ERP and CRM business systems, and the event monitoring component is deployed in the business system. The event monitoring component uses regular expressions to match keywords of business changes such as adding new product lines and adjusting customer classifications to extract event context including change time, involved systems, and affected business objects. After the event context is parsed by the event listening component, it is pushed to the demand processing engine through the Kafka message queue; manually submitted extension requirements are filled in through the web form interface of the management console. The form fields include the extension object, the name of the new attribute, and the category adjustment range. After submission, it is converted into a structured extension requirement description in JSON format and pushed to the demand processing engine through the same Kafka message queue.

[0012] Furthermore, in the master data management method based on enterprise data governance of the present invention, core attributes are basic fields describing master data objects in the basic model structure, including customer ID and name; The classification system is a hierarchical division of master data objects in the basic model structure, including customer type and industry hierarchy; The unique identification rule defines the generation logic of the unique identification of the master data object in the basic model structure, including the customer ID generation algorithm; The element types of the metadata descriptor correspond to the components of the basic model structure, and the constraints include attribute mandatory and length-limited fields; The model warehouse uses MySQL database transactions to write basic model metadata, which is used to query historical versions or real-time effective versions by model name and version number.

[0013] Furthermore, in the master data management method based on enterprise data governance of the present invention, the API gateway is used to obtain the field definition of the real-time master data interface; The interface field extension script modifies the interface return fields and updates the OpenAPI document based on the newly added attributes of the extended model metadata; Data lineage analysis tools display the topological relationship of master data; The data migration script extracts data from the table and writes it to the new table using a SELECT statement, applying the length constraint of the extended model metadata. The data quality rule base is stored in the database table, including rule types and rule expressions; The conflict report includes the conflict rule ID and the conflict reason, which is used by administrators to review the conflict resolution.

[0014] In a second aspect, the present invention provides a master data management system based on enterprise data governance, which is applied to a master data management method based on enterprise data governance, including: The demand perception module deploys event monitoring components in the business system log, parses change events and manual requirements into structured extended requirement descriptions, and pushes them to subsequent modules through the message queue; Dynamic modeling module, which stores basic model and extended model metadata, supports version management and metadata query, and provides a basic model metadata interface to the extended configuration module; The extended configuration module reads the basic model metadata through a visual interface, supports drag-and-drop attribute expansion, tree diagram classification expansion, and relationship diagram association expansion, generates extended model metadata and stores it in the dynamic modeling module; The adaptation engine module reads the extended model metadata, calls the business system API gateway and data quality rule library, performs interface adaptation, data mapping adaptation and rule conflict detection, and outputs the adaptation results; the lifecycle management module synchronizes the effective extended model metadata to the service layer, cleaning layer and monitoring layer, among which the service layer is used for master data service interface management, the cleaning layer is used for data cleaning processing, and the monitoring layer is used for data quality monitoring, for extended synchronization and monitoring of the entire life cycle of master data, and for monitoring the entire life cycle of master data.

[0015] Beneficial effects of the present invention: The present invention uses a metadata-driven basic model construction mechanism to store the core attributes, classification system, unique identification rules and association relationships of the enterprise's general master data objects in JSON format metadata in a versioned model warehouse, providing a traceable benchmark for expansion, avoiding repeated definition of model frameworks, and shortening expansion preparation time; through the demand perception module, business system change events and manual expansion requirements are integrated, and after converting them into structured descriptions, the attribute, classification and association relationship expansion configuration is completed through a visual interface with zero coding. Compared with the existing manual adjustment model, the present invention significantly improves business response efficiency; through the adaptation engine module, the master data interface is automatically expanded, historical data migration scripts are generated and rule conflicts are detected, and the effective model is synchronized to the service layer, cleaning layer and monitoring layer in combination with the lifecycle management module, realizing end-to-end closed-loop management of the expansion model from definition to application, ensuring the consistency, integrity and traceability of master data when the business changes dynamically, and improving the data's support for business innovation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on the drawings without paying any creative labor.

[0017] Figure 1 A flowchart of a master data management method based on enterprise data governance provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the specific embodiments of the present invention and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. The technical solutions provided by each embodiment of the present invention are described in detail below in conjunction with the drawings. In order to better understand the purpose of the present invention, the present invention is further described in detail below.

[0019] First, see Figure 1 The master data management method based on enterprise data governance provided by the present invention includes: Step 1: Define the basic model structure based on the enterprise's common master data objects. The basic model structure includes core attributes, classification system, unique identification rules, and association relationships. It is stored in the form of metadata in a model repository that supports version management and serves as a benchmark for subsequent expansion. Step 2: Obtain business system change events and manually submitted extension requirements, and convert the events and requirements into structured extension requirements that include event types, target master data models, and extension detail fields. Step 3: Based on the structured extension requirements and the basic model metadata stored in the model warehouse, complete the attribute, classification, and relationship extension configuration of the master data model through a visual interface to generate extended model metadata; Step 4: Perform interface adaptation, data mapping adaptation, and rule conflict detection on the extended model metadata. After verification, mark it as valid. Step 5: Synchronize the effective extended model metadata to the service layer, cleaning layer, and monitoring layer. The service layer is used for master data service interface management, the cleaning layer is used for data cleaning processing, and the monitoring layer is used for data quality monitoring. This is used for extended synchronization and monitoring throughout the entire life cycle of master data.

[0020] The master data management method based on enterprise data governance provided by the present invention realizes the dynamic expansion of the master data model through a phased metadata-driven mechanism. Its core steps and specific implementation methods are as follows: First, define the basic model structure based on the enterprise's common master data objects. These objects typically cover core entities shared across business domains, such as customers, products, and suppliers. The basic model structure must clearly define these entities' core attributes (e.g., basic fields like customer ID and name for customer objects), classification systems (e.g., customer types are categorized as individual customers and corporate customers, and industries are categorized as existing manufacturing and new energy), unique identification rules (e.g., using a "company code + timestamp" algorithm to achieve global uniqueness for customer IDs), and relationships (e.g., defining customer-supplier partnerships). This model structure is stored as a JSON-formatted metadata descriptor in a version-supported model repository (e.g., metadata writes are implemented via MySQL database transactions). The metadata descriptor includes fields such as the model version number (e.g., initial version "1.0"), a parent model reference (marked as "null" if the initial model has no parent model), feature types (attributes / classifications / rules), and constraints (e.g., customer ID is a required field and name length is ≤ 50 characters), providing a traceable baseline for subsequent model expansion.

[0021] Secondly, business system change events and manually submitted extension requests are captured and converted into structured descriptions. Business systems typically include core systems such as ERP and CRM. Event monitoring components are deployed in the ERP's MySQL Binlog and the CRM's API call logs. Regular expressions are used to match business change keywords such as "new product line" and "customer classification adjustment" to extract event context, including the change time, the system involved (such as ERP), and the business object affected (such as product). The management console's web form interface is also exposed to business personnel to accept input from extension objects (such as customer models), new attribute names (such as "product environmental certification number"), and category adjustment scopes (such as "new energy category added to customer industry"). The system then converts the parsed system event context and manual requests into a structured extension request description in JSON format via a Kafka message queue. This description includes fields such as eventType (business change / manual request), targetModel (target master data model, such as "customer model"), and extensionDetails (extension details, such as a list of new attributes), which serve as input for model extensions.

[0022] Third, complete the extension configuration of the master data model according to the structured extension requirements. The system reads the basic model metadata stored in the model warehouse through a visual interface. Based on the basic model's real-time attribute list, classification tree, and relationship diagram, it performs three types of extension operations: In the attribute list area, business personnel drag and drop to add attributes (such as "Product Environmental Certification Number"), enter the attribute name, data type (such as string), and constraint rules (such as length ≤ 20), and associate the enterprise terminology library (such as associating "Environmental Certification Number" by term ID) to generate attribute extension metadata fragments. In the classification system area, a tree diagram displays the current classification of the basic model (such as "Customer Industry → Existing Manufacturing"), and users can right-click a node and select "Add Sub-Classification" (such as "New Energy") to generate classification extension metadata fragments. The classification version (such as upgrading from V1.0 to V1.1) and the old and new mapping rules (such as "Retain the old classification 'Existing Manufacturing' and add 'New Energy'") are recorded to generate classification extension metadata fragments. In the relationship area, a relationship diagram component displays existing relationships (such as "Customer-Supplier"), selects new association objects (such as "Product" and "R&D Team"), and configures rules (such as "The ID of the R&D team to which the product belongs must exist in the R&D team master data") to generate relationship extension metadata fragments. After the extension is completed, the system merges each type of extended metadata fragment with the basic model metadata to generate extended model metadata marked as "pending verification".

[0023] Fourth, interface adaptation, data mapping adaptation and rule conflict detection are performed on the extended model metadata to verify its validity. During the interface adaptation phase, the system calls the API gateway metadata service to obtain the field definitions of the current master data interface (e.g., the GET / customer interface returns customerID and name), parses the newly added attributes of the extended model metadata (e.g., "certificateNo"), automatically generates interface field extension scripts (e.g., modifying the interface return fields to customerID, name, and certificateNo), and updates the OpenAPI document. During the data mapping adaptation phase, data lineage analysis tools such as Apache Atlas are used to track the storage location of historical master data instances (e.g., the Hive table customer_v1), extract historical fields related to extended attributes (e.g., "certification remarks"), and generate data migration scripts based on the attribute constraints of the extended model (e.g., certificateNo length ≤ 20 characters). This includes extracting the first 20 digits of the remark field from the customer_v1 table and writing them into the new table customer_v2. During the rule conflict detection phase, the system reads the data quality rule library (stored in the database table quality_rules, including rules such as "customerIDUNIQUE") to verify whether the newly added attributes conflict with existing rules (e.g., if "certificateNo UNIQUE" is added, it is necessary to check whether it conflicts with "productCode"). UNIQUE" is duplicated), a report including the conflicting rule ID and reason (for example, {"ruleID":"R001","reason":"The uniqueness rule for certificateNo and productCode is duplicated"}) is generated and pushed to the console for review. If no conflict is detected or the conflict has been resolved, the extended model metadata is marked as "Effective".

[0024] Finally, the effective extended model metadata is synchronized to the service, cleaning, and monitoring layers for full lifecycle management. The service layer publishes the extended interface field definitions (e.g., a client interface including certificateNo) via a message bus like RabbitMQ, notifying the business system to subscribe to the model_update topic and triggering local interface adaptation (e.g., updating the interface return fields). The cleaning layer uses a data cleaning engine like Apache Spark to read the extended model's attribute constraints (e.g., certificateNo length ≤ 20) and writes the rules into the cleaning task configuration (e.g., WHERE LENGTH(certificateNo) <= 20). These rules are automatically applied to newly added or updated master data instances. The monitoring layer extracts extended attributes (e.g., certificateNo) from the model repository and adds a "certificateNo missing rate" metric (e.g., metric ID M005) to the monitoring metric configuration table. A scheduled task (e.g., at midnight daily) collects data on the certificateNo field in master data instances, calculates the missing rate, and displays it on the dashboard. After synchronization is complete at all layers, the system records the final effective status of the extended model, forming a complete closed loop from demand perception to extension implementation.

[0025] The above steps achieve full-process management of the master data model from basic definition to dynamic expansion through metadata-driven process connection, improving the sharing efficiency and business support capabilities of master data.

[0026] Specifically, the master data management method based on enterprise data governance of the present invention obtains business system change events by deploying an event monitoring component for log collection in the database change log and API call log of the business system, matching business change keywords through regular expressions, and extracting event context including change time, involved systems, and affected business objects; Obtaining manually submitted extension requirements includes receiving the extension object, new attribute name, and category adjustment range filled in by business personnel through the management console web form interface open to business personnel; The event context extracted by the event listening component and the manually submitted extension requirements are uniformly converted into structured extension requirements for the event type, target master data model, and extension detail fields in step 2 through the message queue.

[0027] To capture business system change events, an event monitoring component deployed in database change logs and API call logs operates as a lightweight agent. This component, pre-installed with a regular expression matching rule library, continuously scans log content. Upon detecting a log entry containing predefined business change keywords such as "new product line" or "adjusted customer category," it automatically extracts a triplet of information: the precise timestamp of the change, the name of the source business system module involved, and the type of business object affected. For example, when a table structure change statement is identified in the MySQL Binlog of an ERP system, the event monitoring component marks the operation time as the event triggering moment, recording the system module as "Procurement Management" and the business object as "Product Master Data."

[0028] Manual extension requests are collected through an interactive web form interface. The management console provides a tree selector for business personnel to select the target master data model. An input box receives the name of the newly added attribute and automatically verifies the naming standardization by linking it to the corporate terminology database. The hierarchy selector specifies the parent category node for category adjustment and enters the name of the newly added subcategory. After the form is submitted, the backend service verifies the logical integrity of the input data, including the compliance of attribute data types and the validity of the classification hierarchy, converting the manual operation intent into a standard data format.

[0029] Event context and manual requirements are formatted uniformly through distributed messaging middleware. The context triplet output by the event listener and the manually submitted extension parameters are encapsulated into a JSON structure and sent to the message queue via the Kafka producer. This structure includes an event type identifier, the name of the target master data model, and a detailed list of extended operations. The demand processing engine, acting as a consumer, subscribes to message topics and parses heterogeneous input into a unified format, including an event type field, a model identifier field, and a list of extended fields.

[0030] The logical connection between these steps is as follows: the event monitoring component captures business system change signals in real time; the management console provides a standardized input channel for manual requirements; the message queue mechanism solves the transmission issues of heterogeneous data sources; and finally, a standardized requirement description is output for subsequent extension configuration modules to call. Technical terminology remains consistent: "event monitoring component" refers to the log scanning agent, "management console" refers to the web interface, and "structured extension requirement" refers to a JSON data object with fixed fields.

[0031] Specifically, the master data management method based on enterprise data governance of the present invention has a basic model structure that is a model framework based on the definition of enterprise common master data objects, specifically including core attributes, classification system, unique identification rules and association relationships; The basic model structure is stored in the form of a metadata descriptor in JSON format. The metadata descriptor includes the model version number, the parent model reference, the type field that maps the basic model structure elements, and the constraint condition field corresponding to the unique identification rules and association relationships; The model warehouse stores basic model metadata including historical versions through database transactions.

[0032] The definition, storage, and management of the basic model structure in this invention are the cornerstones of dynamic expansion of master data. The specific implementation methods and logical relationships are as follows: The basic model structure is based on the definition of common enterprise master data objects and covers core entities shared across business areas. Common enterprise master data objects typically include frequently used business entities such as customers, products, and suppliers. For example, customer master data objects describe the interactions between an enterprise and external individuals or organizations, while product master data objects define the attributes of goods or services provided by the enterprise. The basic model structure must clearly define the core components of these entities: core attributes are fundamental fields that describe the essential characteristics of the object, such as the customer object's "Customer ID" (a unique identifier field) and "Name" (a business identifier field); the classification system is a hierarchical division of objects, such as the customer type classification of "individual customer" and "corporate customer" and the industry classification of "existing manufacturing" and "new energy"; unique identification rules are the logic for generating objects that are globally unique, such as the customer ID using a combination of "enterprise code (6 digits) + timestamp (14 digits)"; and relationships are the business connection rules between objects, such as the definition of a "customer-supplier" relationship that states "the customer's primary supplier ID must exist in the supplier master data."

[0033] The basic model structure is stored as a metadata descriptor in JSON format, making model elements machine-readable and standardized. The metadata descriptor includes the following core fields: the model version number (e.g., "1.0" for the initial version, "1.1" for expansion and upgrade), which identifies the iteration status of the model; the parent model reference ("null" for the initial model, and "1.0" for the expanded model), which traces the inheritance relationship of the model; the feature type (distinguishing between "attribute," "category," and "rule"), which categorizes the model feature; and constraints (e.g., "required:true" for required attributes and "maxLength:50" for attribute length limits), which define the business rules for model features. The model repository stores and manages basic model metadata through database transactions, providing a traceable baseline for subsequent expansion. The model repository uses a MySQL database and implements atomic metadata writes through transaction operations (e.g., BEGIN...COMMIT) to prevent data inconsistencies. The warehouse supports querying historical or currently active versions by model name (such as "customer_master") and version number (such as "1.0"). For example, this allows obtaining metadata for the underlying model of customer master data. Versioning allows developers to view model iterations (such as the difference between "1.0" and "1.1"). Metadata traceability links extended models to the base model through a parent model reference field (such as "parentModel: '1.0'"), ensuring auditability of extension operations.

[0034] The above design builds a benchmark framework for master data model extension through element definitions of specific entities, standardized metadata descriptions, and versioned warehouse management. This allows subsequent model extensions to be performed based on a clear metadata foundation, ensuring the standardization and traceability of extension operations.

[0035] Specifically, the master data management method based on enterprise data governance of the present invention further includes: reading the defined basic model metadata from the model warehouse, and based on the real-time attribute list, classification system and association relationship of the basic model, including: Add attributes to the attribute list area by dragging and dropping, enter the attribute name, data type, and constraint rules, associate the enterprise terminology base, and generate attribute extended metadata fragments; In the classification system area, the real-time classification system of the basic model is displayed in a tree diagram. Users can right-click and select nodes to add sub-categories, record classification versions and mapping rules between old and new classifications, and generate classification extension metadata fragments. In the relationship area, use the relationship diagram component to display the existing relationships of the basic model, select new relationship objects, configure the relationship rules, and generate the relationship metadata fragment; After the extension is completed, the extended metadata fragments of attributes, classifications, and associations are merged with the basic model metadata to generate extended model metadata marked as pending verification.

[0036] The visual interface extension configuration in the present invention is the core interactive link for realizing the zero-coding extension of the master data model. Its specific implementation and logical relationship are as follows: The visualization interface first reads the base model metadata from the model repository as the initial basis for extended configuration. The system then calls the model repository's metadata query interface (e.g., GET / model / {model_id} / base) to retrieve the base model's real-time attribute list, classification system, and relationship data. For example, for a customer master data model, the attribute list read might include fields such as "customerID" and "name." The classification system is presented in a tree structure as "Customer Type → Individual Customer / Corporate Customer" and "Industry → Existing Manufacturing / New Energy." Relationships are displayed in a diagram as connections such as "Customer-Supplier" and "Customer-Product." This data is rendered into a visualization interface using front-end components (such as React's TreeView and GraphVis) for intuitive operation by business personnel.

[0037] Attribute extension is completed in the attribute list area by dragging and dropping. The "Add Attribute" component library is provided on the left side of the interface. After the business personnel drags the component to the attribute list area, an input box pops up to require the attribute name (such as "Environmental Certification Number"), data type (select "String", "Integer", etc. through the drop-down menu) and constraint rules (such as "Length ≤ 20" and "Required"). The system automatically calls the enterprise terminology library interface (such as GET / term / {term_id}) to verify the standardization of the attribute name (such as whether it is a defined standard term). After the verification is passed, the attribute extension metadata fragment is generated. For example, when adding the "Environmental Certification Number" attribute to the product master data model, the system generates the corresponding attribute extension fragment and temporarily stores it in the front-end cache.

[0038] The classification system is expanded through a tree-based interactive interface. This interface displays the real-time classification system of the underlying model in a tree diagram (e.g., "Customer Industry → Existing Manufacturing"). Business personnel right-click the target node (e.g., "Existing Manufacturing"), select "Add Subcategory," and enter the new category name (e.g., "New Energy"). The system automatically records classification version changes (e.g., upgrading from V1.0 to V1.1) and generates a mapping rule between the old and new categories (e.g., "Retain the old category 'Existing Manufacturing' and add 'New Energy' as its child"), generating a classification expansion metadata segment {"type":"category","name":"New Energy","parent":"Existing Manufacturing","version":"1.1"}. For example, when a "New Energy" subcategory is added to the customer industry classification, the system simultaneously updates the classification tree structure and generates the corresponding classification expansion segment.

[0039] Relationship expansion is configured through the relationship graph component. The interface displays the existing relationships of the basic model in the form of a relationship graph (for example, the nodes "Customer" and "Supplier" are connected through a "Partnership"). Business personnel drag new related objects (such as "Product" and "R&D Team") from the object library to the relationship graph area, select the connection edge, and configure the association rules (such as "The ID of the R&D team to which the product belongs must exist in the R&D team master data"). The system verifies the validity of the rules (for example, checking whether the "R&D Team ID" is a valid field in the R&D team master data) and generates a relationship expansion metadata fragment. For example, when adding an association with the R&D team to the product master data model, the system generates a corresponding relationship expansion fragment.

[0040] After the extension is completed, the system merges the extended metadata fragments of attributes, classifications, and associations with the base model metadata. The front-end component calls the back-end merge interface (such as POST / model / {model_id} / merge) to add each fragment to the "elements" list of the base model metadata to generate complete extended model metadata. At the same time, the system updates the status field of the model in the metadata database to "pending verification" and records the timestamp of the extension operation and the operator ID (such as "operator: user001, timestamp: 2025-05-27 15:30:00"). For example, after the customer master data model completes the attribute, classification, and association extensions, the merged extended model metadata includes the original elements of the base model and the newly added extended fragments, and the status is marked as "pending verification", awaiting subsequent interface adaptation and rule detection.

[0041] Through interactive operations on a visual interface, the above steps convert business personnel's expansion requirements into structured metadata fragments, which are ultimately merged to generate verifiable expansion model metadata. This achieves zero-coding expansion of the master data model, lowers the technical threshold for model adjustment, and improves the efficiency of responding to business needs.

[0042] Specifically, the master data management method based on enterprise data governance of the present invention also includes: Call the business system API gateway metadata service to obtain the field definition of the real-time master data interface, parse the newly added attributes in the extended model metadata, automatically generate the interface field extension script, and update the OpenAPI document of the master data service interface; Use data lineage analysis tools to track the storage locations of historical master data instances, extract historical fields related to extended attributes, and generate data migration scripts based on attribute constraints of extended model metadata. Read the data quality rule library, verify whether the newly added attributes of the extended model metadata conflict with existing rules, generate a report including the conflict rule ID and the cause of the conflict, and push it to the administrator console for review; If no conflict is detected or the conflict has been resolved, the extended model metadata is marked as valid.

[0043] In the present invention, interface adaptation, data mapping adaptation, and rule conflict detection are key steps in verifying the business adaptability of extended model metadata. The specific implementation methods and logical relationships are as follows: During the interface adaptation phase, the master data interface is automatically extended through the API Gateway metadata service. The system calls API Gateway metadata services such as Swagger Hub to obtain the current master data interface's real-time field definitions (e.g., the return fields for the GET / customer interface are ["customerID", "name"]). After parsing the newly added attributes in the extended model metadata (e.g., extracting the "certificateNo" attribute through JSON fields), the system automatically generates an interface field extension script, for example, modifying the interface's return fields to ["customerID", "name", "certificateNo"]. After the script executes, the system simultaneously updates the master data service interface's OpenAPI documentation (e.g., the "responses" section in the Swagger documentation), allowing business systems to easily access the latest interface specifications through the documentation. For example, when adding the "Environmental Certification Number" attribute to the customer master data interface, the interface adaptation operation expands the original interface's return fields, allowing ERP and CRM systems calling the interface to retrieve the complete data, including the new attribute.

[0044] During the data mapping adaptation phase, data lineage analysis tools are used to ensure compatibility between historical data and the extended model. The system uses data lineage analysis tools such as Apache Atlas to track the storage locations of historical master data instances (e.g., the customer_v1 table in the Hive data warehouse) and extract historical fields related to extended attributes (e.g., the "Certificate Remarks" field in the customer_v1 table). Based on attribute constraints defined in the extended model metadata (e.g., "certificateNo" length ≤ 20 characters), the system generates a data migration script. For example, using the SQL statement "INSERT INTO customer_v2 (customerID, name,certificateNo) SELECT customerID, name, SUBSTR(remark, 1, 20) FROM customer_v1," the system truncates the first 20 characters of the "Certificate Remarks" field in the historical data and maps them to the new "certificateNo" attribute. This operation ensures compatibility between historical data and the extended model structure, avoiding data gaps caused by the model expansion.

[0045] During the rule conflict detection phase, the compliance of the extended model is verified through the data quality rule library. The system reads the data quality rules stored in the database table quality_rules (such as "customerID UNIQUE" and "name NOT NULL") and verifies whether the newly added attributes of the extended model conflict with existing rules. For example, if a new attribute definition "certificateNoUNIQUE" is added, the system checks whether other fields in the quality_rules table (such as "productCode") already have the same type of uniqueness rule defined. If a conflict is detected (such as the duplication of the "productCode UNIQUE" and "certificateNoUNIQUE" rules), the system generates a report including the conflict rule ID (such as "R001") and the cause of the conflict (such as "The uniqueness rules for certificateNo and productCode are duplicated") and pushes it to the administrator console. Based on the report, the administrator can adjust the constraint rules of the extended model (such as canceling the definition of "certificateNo UNIQUE") or modify existing rules (such as adjusting the scope of "productCode UNIQUE").

[0046] If no conflicts are detected or have been manually resolved, the system updates the status field of the extended model metadata from "Pending Verification" to "Effective." This metadata will serve as the basis for subsequent synchronization of the service layer, cleaning layer, and monitoring layer, allowing the extended master data model to be officially put into business use.

[0047] Through the orderly execution of interface expansion, data migration, and rule verification, the above steps verified the feasibility of extending the model metadata in terms of technical implementation, data compatibility, and business compliance, providing multi-dimensional guarantees for the formal implementation of the model.

[0048] Specifically, the master data management method based on enterprise data governance of the present invention further includes: publishing the expanded master data service interface field definition via a message bus, notifying the business system to subscribe to the topic, and triggering the business system local interface adaptation; The data cleaning engine reads the attribute constraint rules of the effective extended model metadata from the model warehouse, writes the rules into the cleaning task configuration, and automatically applies the rules to newly connected or updated master data instances; Extract the extended attributes of the effective extended model metadata from the model warehouse, add new records to the monitoring indicator configuration table, collect field data of the master data instance through scheduled tasks, calculate the missing rate and display it on the dashboard; after the synchronization of each layer is completed, the system records the final effective status of the extended model.

[0049] In this invention, the synchronization of the service layer, cleaning layer, and monitoring layer is a key step in the implementation of the extended model metadata after it takes effect. Each layer is used for dynamic management of the entire life cycle of master data through parallel execution. The specific implementation methods and logical relationships are as follows: The service layer simultaneously publishes the expanded interface definition via the message bus, triggering business system adaptation. Based on the effective extended model metadata (e.g., a customer master data model that includes the newly added attribute "certificateNo"), the system publishes a model_update topic message via a message bus like RabbitMQ. The message content includes the expanded master data service interface field definition (e.g., the return fields of the GET / customer interface are expanded from ["customerID", "name"] to ["customerID", "name", "certificateNo"]). Business systems (such as ERP and CRM) subscribe to this topic to receive interface change notifications and invoke local interface adaptation scripts (e.g., modifying the API client's response parsing logic) to correctly receive and process the newly added field data. For example, after subscribing to the model_update topic, an ERP system detects the addition of the "certificateNo" field to the customer interface and automatically updates the customer data display interface in its procurement module to add a display column for this field.

[0050] The cleaning layer simultaneously writes extended attribute constraint rules through the data cleaning engine to ensure data quality. The system invokes data cleaning engines such as Apache Spark to read attribute constraint rules (such as "certificateNo length ≤ 20") from the model repository's effective extended model metadata. These rules are converted into filter conditions for the cleaning task (such as WHERE LENGTH(certificateNo) ≤ 20) and written into the task configuration file. New master data instances (such as customer data imported from business systems via ETL tools) or updated instances (such as data submitted by customers modifying their environmental certification information) are automatically applied to these rules before entering the master data warehouse, filtering or correcting data that does not comply with the constraints (for example, truncating the "certificateNo" field to the first 20 digits if it exceeds 20 digits). For example, if a customer submits a "certificateNo" value of "CN-20250527-ABCDEFGHIJKLMN" (length 24), the cleaning task will automatically truncate it to "CN-20250527-ABCDEFGHIJKL" (length 20), ensuring that the data complies with the model constraints.

[0051] The monitoring layer simultaneously implements data quality tracking for extended attributes through new monitoring indicators. The system extracts extended attributes (such as "certificateNo") from the metadata of effective extended models from the model repository and creates new records in the monitoring indicator configuration table (such as monitoring_metrics), including the metric name (such as "certificateNo missing rate") and display dimensions (such as by customer type or industry classification). A scheduled task (such as a daily trigger at midnight) collects data from the "certificateNo" field of master data instances, calculates the missing rate, and stores it in a monitoring database (such as InfluxDB). A monitoring dashboard (such as Grafana) queries the monitoring database and displays a real-time trend chart of the "certificateNo missing rate." Business personnel can use this metric to assess the quality of extended attribute population (for example, a missing rate of 15% in a given week indicates a need to strengthen data entry standards).

[0052] After synchronization of all layers is complete, the system updates the status field of the extended model metadata in the model repository to "Final," records the synchronization completion timestamp (e.g., "2025-05-27 16:00:00"), and records the business systems involved (e.g., "ERP Procurement Module, CRM Customer Management Module"). This status update indicates that the extended model has been fully integrated into the enterprise master data management process. Subsequent business operations will use this model for data exchange, cleansing, and monitoring.

[0053] The above steps are carried out in parallel through service layer interface adaptation, cleaning layer rule application, and monitoring layer indicator tracking, so that the expanded master data model can be implemented simultaneously at the technical implementation, data quality, and business monitoring levels, forming a complete closed loop from model expansion to business application.

[0054] Specifically, the master data management method based on enterprise data governance of the present invention includes business systems including ERP and CRM business systems. The event monitoring component is deployed in the MySQL Binlog log of ERP and the API call log of CRM. Regular expressions are used to match keywords of business changes such as adding new product lines and adjusting customer classifications to extract event context including change time, involved systems, and affected business objects. After the event context is parsed by the event listening component, it is pushed to the demand processing engine through the Kafka message queue. Manually submitted extension requirements are filled in through the web form interface of the management console. The form fields include the extension object, the name of the new attribute, and the category adjustment range. After submission, the Spring Boot backend service converts it into a structured extension requirement description in JSON format and pushes it to the demand processing engine through the same Kafka message queue.

[0055] The acquisition and integration of business system change events and manual expansion requirements in the present invention is the input link for realizing the dynamic expansion of the master data model. The specific implementation method and logical relationship are as follows: Business system change events are captured by deploying event listeners in the logs of core business systems. Core business systems typically include ERP (Enterprise Resource Planning) and CRM (Customer Relationship Management). Event listeners should be deployed in the log sources of these two systems: For ERP systems, database change logs (such as MySQL Binlog) are used, which record detailed information about all table structure changes (such as ALTER TABLE operations) and data changes (such as INSERT / UPDATE statements). For CRM systems, API call logs (such as Spring Boot-based REST API access logs) are used, which record the request parameters and timestamps of business operations (such as customer category modifications and supplier information updates) triggered by business personnel through the front-end interface. For example, when the ERP system executes the table structure change "ALTER TABLE product ADDCOLUMN new_feature VARCHAR(50)", the MySQL Binlog log will record the time of the operation (such as 2025-05-20 14:30:00), the table name involved (product), and the newly added field (new_feature). The event monitoring component captures the change event by monitoring the log source.

[0056] The event listening component extracts business change keywords and contextual information through regular expression matching. The component pre-defines regular expression patterns to match business change keywords related to master data model expansion. For ERP database change logs, it matches keywords such as "ADD COLUMN" and "ALTER TABLE" to extract the change time (such as the "timestamp" field in the log), the involved system (marked as "ERP"), and the affected object (such as the "product" master data model). For CRM API call logs, it matches interface parameter keywords such as "update_category" and "add_attribute" to extract the change time (such as the "request_time" field in the request), the involved system (marked as "CRM"), and the affected object (such as the "customer" master data model). For example, when the request "POST / api / customer / update?action=add_category&category=New Energy" appears in the CRM system API call log, the event listening component matches the "add_category" keyword and extracts the event context as "Change time: 2025-05-20 15:00:00, Involved system: CRM customer management module, Affected object: customer master data model".

[0057] The parsed event context is pushed to the demand processing engine via the Kafka message queue. The event listener encapsulates the extracted system event context (such as an ERP table structure change event) in JSON format (e.g., {"eventType":"system","source":"ERP","targetModel":"product","details":"ADD COLUMN new_feature"}) and sends it to the "model_extension_request" topic via the Kafka producer. The demand processing engine (such as a Spring Kafka-based consumer) subscribes to this topic, receiving and parsing system event messages in real time as input for model extensions.

[0058] Manually submitted extension requests are collected and converted through the management console's web form interface. Business personnel access the master data management system's management console through a browser and enter the "Model Extension" page. This page provides a visual form with the following fields: the extension object (select a master data model such as "customer," "product," or "supplier" from the drop-down menu), the name of the new attribute (which must comply with the company's terminology, such as "Environmental Certification Number"), and the scope of the category adjustment (select a parent category in a tree structure and enter a subcategory name, such as "Customer Industry → Existing Manufacturing → New Energy"). After the form is submitted, the Spring Boot backend service validates the input: attribute names are verified for standard terminology using the terminology interface (e.g., GET / term / validate?name=Environmental Certification Number). Category adjustments also check the existence of the parent category (e.g., "Existing Manufacturing" is an existing category for the customer industry). Once verified, the backend converts the manual request into JSON format and sends it to the "model_extension_request" topic via the same Kafka producer, where it is merged with system event messages.

[0059] The demand processing engine receives and integrates system events and manual request messages, triggering the subsequent model extension process. Through Kafka message queue integration, system events and manual requests are delivered to the demand processing engine in the same format (JSON) and the same topic (model_extension_request). The engine distinguishes the message source based on the eventType field, extracts the targetModel and details fields, and invokes the model extension configuration module for subsequent operations.

[0060] The above steps transform scattered business change signals into machine-readable structured requirements through log monitoring, keyword matching, message queue integration, and manual demand standardization, providing a clear and unified input basis for the subsequent dynamic expansion of the master data model.

[0061] Specifically, in the master data management method based on enterprise data governance of the present invention, the core attributes are the basic fields describing the master data objects in the basic model structure, including customer ID and name; The classification system is a hierarchical division of master data objects in the basic model structure, including customer type and industry hierarchy; The unique identification rule defines the generation logic of the unique identification of the master data object in the basic model structure, including the customer ID generation algorithm; The element types of the metadata descriptor correspond to the components of the basic model structure, and the constraints include attribute mandatory and length-limited fields; The model warehouse uses MySQL database transactions to write basic model metadata, which is used to query historical versions or real-time effective versions by model name and version number.

[0062] The core components of the basic model structure and the storage management mechanism in this invention are the underlying support for the expansion of the master data model. The specific implementation methods and logical relationships are as follows: Core attributes serve as the fundamental descriptive fields for master data objects, clarifying the essential characteristics of the objects. For example, the core attributes of customer and product master data objects, commonly found in enterprises, include "Customer ID" (a globally unique identifier, such as "CUST-20250527-001") and "Name" (a customer's business identifier, such as "XX Technology Co., Ltd."); the core attributes of product master data include "Product ID" (such as "PROD-20250527-001") and "Product Name" (such as "New Energy Vehicle Battery"). These attributes serve as the fundamental identifiers for master data objects in cross-system interactions, ensuring consistent identification of the same object in different business scenarios. For example, an ERP system associates purchase orders with "Customer ID," while a CRM system associates customer service records with "Customer ID." These two systems communicate data through this core attribute.

[0063] The classification system implements business categorization of master data objects through hierarchical divisions. Taking customer master data as an example, the classification system includes two dimensions: "Customer Type" and "Industry." "Customer Type" serves as the first-level classification, divided into "Individual Customer" and "Corporate Customer." "Corporate Customer" can be further subdivided into "Large Enterprise" and "Small and Medium-Sized Enterprise" (secondary classifications). "Industry" serves as another first-level classification, divided into "Existing Manufacturing," "New Energy," and "Information Technology." "Existing Manufacturing" can be further subdivided into "Automotive Manufacturing" and "Mechanical Processing" (secondary classifications). This hierarchical structure enables business personnel to filter and analyze data based on different dimensions. For example, the marketing department can quickly identify target customer groups by using "Customer Type = Corporate Customer" and "Industry = New Energy."

[0064] Unique identification rules ensure global uniqueness for master data objects through explicit generation logic. For example, the customer ID generation algorithm uses a combination of "enterprise code (6 digits) + timestamp (14 digits) + serial number (3 digits)": The enterprise code is assigned upon enterprise registration (e.g., "ABC001"), the timestamp is the current time with millisecond accuracy (e.g., "20250527153045000"), and the serial number is used to handle multiple object creation within the same millisecond (e.g., "001"). This algorithm ensures that each customer ID is unique within the enterprise, preventing data confusion caused by duplicate IDs. For example, if two customers are created within the same millisecond, with serial numbers "001" and "002," the generated customer IDs are "ABC00120250527153045000001" and "ABC00120250527153045000002," respectively, ensuring uniqueness.

[0065] The metadata descriptor maps the components of the underlying model through standardized fields and defines business rules. The "element type" field of the metadata descriptor corresponds one-to-one with the components of the underlying model: the "attribute" type corresponds to the core attributes (such as "customerID" and "name"), the "category" type corresponds to the classification system (such as "customer type" and "industry"), and the "rule" type corresponds to the unique identification rule (such as "customer ID generation algorithm"). The constraint field defines the business rules of the attribute. For example, the constraint for "customerID" is {"required": true} (required), and the constraint for "name" is {"maxLength": 50} (length ≤ 50). For example, the metadata descriptor fragment of customer master data can be expressed as: The model repository uses MySQL database transactions to reliably store and query metadata. The model repository's metadata table (e.g., model_metadata) includes fields such as "model_name" (model name, such as "customer_master"), "model_version" (model version, such as "1.0"), and "metadata_json" (a JSON string representing the metadata descriptor). Database transactions (BEGIN...COMMIT) are used to ensure atomicity when writing data. For example: Query support is available by model name and version number. For example, query "SELECT metadata_json FROMmodel_metadata WHERE model_name='customer_master' AND model_version='1.0'" to retrieve the base model metadata for customer master data. Versioning allows tracking model iteration history (e.g., changes from "1.0" to "1.1"). Metadata traceability links extended models to base models using a parent model reference field (e.g., "parentModel": "1.0"), ensuring auditability of extension operations.

[0066] The above design builds a benchmark framework for the expansion of the master data model through clear identification of core attributes, hierarchical management of the classification system, algorithmic definition of uniqueness rules, standardized description of metadata and versioned storage of the warehouse, providing a standardized and traceable basis for the dynamic adjustment of subsequent models.

[0067] Specifically, in the master data management method based on enterprise data governance of the present invention, the API gateway element is used to obtain the field definition of the real-time master data interface; The interface field extension script modifies the interface return fields and updates the OpenAPI document based on the newly added attributes of the extended model metadata; Data lineage analysis tools display the topological relationship of master data; The data migration script extracts data from the table and writes it to the new table using a SELECT statement, applying the length constraint of the extended model metadata. The data quality rule base is stored in the database table, including rule types and rule expressions; The conflict report includes the conflict rule ID and the conflict reason, which is used by administrators to review the conflict resolution.

[0068] The collaborative application of the API gateway metadata service, data lineage analysis tools, and data quality rule base in this invention is the technical support link for verifying the business adaptability of the extended model. Its specific implementation methods and logical relationships are as follows: The API Gateway metadata service implements real-time field retrieval for master data interfaces through Swagger Hub. The master data management system calls the Swagger Hub API (e.g., GET / apis / {api_id} / metadata) to retrieve the OpenAPI document for the current master data service interface. This document includes the interface path (e.g., GET / customers), HTTP method (e.g., GET), and response field definitions (e.g., the "schema" attribute in the "responses" section). For example, in the OpenAPI document for the customer master data interface, the system parses the "properties" field to retrieve the real-time field list returned by the current interface (e.g., ["customerID", "name"]), which serves as the basis for interface expansion.

[0069] The interface field extension script dynamically adjusts the interface definition based on the extended model metadata. The system parses the newly added attributes in the extended model metadata (such as "certificateNo", with a data type of "string" and a constraint of "maxLength: 20") and generates the interface field extension script. The script modifies the "properties" section of the OpenAPI document to add the definition of the attribute: After the script executes, the system calls the Swagger Hub's update API to synchronize the modified OpenAPI document to the gateway, allowing business systems to view the latest interface specifications in real time through tools such as the Swagger UI. For example, the expanded customer interface now includes a new response field called "certificateNo," allowing the ERP system to retrieve complete customer data when calling this interface.

[0070] The data lineage analysis tool uses Apache Atlas to track the storage and association of historical data. The system calls Apache Atlas's lineage query API (e.g., GET / v2 / lineage / {guid}) to obtain the storage path and field associations of historical master data instances. For example, historical instances of product master data are stored in the product_v1 table in the Hive data warehouse. Apache Atlas records the lineage information for this table (e.g., data originating from the procurement module of an ERP system, including the "productID," "name," and "remark" fields). By parsing this lineage information, the system extracts historical fields related to newly added attributes (e.g., the "remark" field, which is used to map to the newly added "certificateNo" attribute) and obtains the storage location of these fields (e.g., "product_v1.remark" in the Hive table).

[0071] The data migration script uses SQL statements to ensure compatibility between historical data and the extended model. The system generates the data migration script based on the length constraints of the extended model metadata (for example, the length of "certificateNo" must be ≤ 20 characters). The script uses SELECT statements to extract relevant fields from the historical table and applies the constraints.

[0072] The data quality rule library stores business rules in database tables, supporting conflict detection and report generation. Rule library tables (such as quality_rules) include fields such as "rule_id" (a unique identifier for a rule, such as "R001"), "rule_type" (rule type, such as "UNIQUE" or "NOT_NULL"), and "rule_expression" (rule expression, such as "customerID UNIQUE"). The system accesses the rules in this table and verifies whether newly added attributes of the extended model conflict with existing rules. For example, if a new attribute is defined as "certificateNo UNIQUE," the system queries all "UNIQUE"-type rules in the rule library to check whether the same rule already exists for another field (such as "productCode"). If a conflict is detected (e.g., "productCodeUNIQUE" and "certificateNo UNIQUE" are duplicates), the system generates a conflict report and pushes it to the administrator console. Administrators can use the report to adjust the constraints of the extended model (e.g., remove the definition of "certificateNo UNIQUE") or modify existing rules (e.g., adjust the scope of "productCode UNIQUE").

[0073] The above steps achieve multi-dimensional verification of the extended model in terms of technical implementation, data compatibility, and business compliance through interface expansion of the API gateway, historical tracking of data lineage, constraint application of migration scripts, and conflict detection of the rule base, providing a reliable basis for the formal implementation of the model.

[0074] In a second aspect, the present invention provides a master data management system based on enterprise data governance, which is applied to a master data management method based on enterprise data governance, including: The demand perception module deploys event monitoring components in the business system log, parses change events and manual requirements into structured extended requirement descriptions, and pushes them to subsequent modules through the message queue; Dynamic modeling module, which stores basic model and extended model metadata, supports version management and metadata query, and provides a basic model metadata interface to the extended configuration module; The extended configuration module reads the basic model metadata through a visual interface, supports drag-and-drop attribute expansion, tree diagram classification expansion, and relationship diagram association expansion, generates extended model metadata and stores it in the dynamic modeling module; The adaptation engine module reads the extended model metadata, calls the business system API gateway, data lineage analysis tools and data quality rule library, performs interface adaptation, data mapping adaptation and rule conflict detection, and outputs the adaptation results; the lifecycle management module synchronizes the effective extended model metadata to the service layer, cleaning layer and monitoring layer, among which the service layer is used for master data service interface management, the cleaning layer is used for data cleaning processing, and the monitoring layer is used for data quality monitoring, for extended synchronization and monitoring of the entire life cycle of master data, and for monitoring the entire life cycle of master data.

[0075] The master data management system based on enterprise data governance provided by this invention implements the full-process management of the master data model from demand perception to expansion and implementation through modular design. Each module works together. Its specific implementation method and logical relationship are as follows: Demand perception module: Capture and integrate business change signals The demand perception module is the input entry of the system. It is responsible for capturing dynamic change events of the business system and manually submitted expansion requirements, and converting them into structured data for subsequent modules to process. This module is implemented in the following ways: Event monitoring component deployment: Deploy monitoring components in the log sources of core enterprise business systems (such as ERP and CRM). These monitoring components include the ERP system's MySQL Binlog (recording database table structure and data changes) and the CRM system's API call log (recording business operation request parameters). The component uses regular expressions to match business change keywords such as "ADD COLUMN" and "update_category" to extract event context (such as change time, affected systems, and affected objects). For example, it can parse the ERP Binlog to extract the event information "2025-05-20 14:30:00, ERP procurement module, new field added to product master data."

[0076] Manual Requirements Collection: A web form interface (developed with the Vue front-end framework) is provided for the management console. Business personnel can enter extended objects (such as "Customer Master Data Model"), new attribute names (such as "Environmental Certification Number"), and category adjustment ranges (such as "New Energy in Customer Industry"). After the form is submitted, the Spring Boot backend service performs terminology compliance verification on the input (using the enterprise terminology library interface to verify attribute names) to ensure that the requirements meet business standards.

[0077] Structured requirement output: System event context and manual requirements are uniformly converted into JSON format through the Kafka message queue (topic "model_extension_request"), including the eventType (system / manual), targetModel (target model), and extensionDetails (extension details) fields, and pushed to subsequent modules.

[0078] Dynamic Modeling Module: Storing and managing model metadata The dynamic modeling module is the core metadata storage unit of the system. It is responsible for the versioning management of the basic model and the extended model, and provides benchmark data for the extended configuration. This module is implemented in the following ways: Metadata storage: A MySQL database is used to store metadata. The table structure includes fields such as model_name (model name, such as "customer_master"), model_version (version number, such as "1.0"), and metadata_json (metadata descriptor in JSON format). Database transactions (BEGIN...COMMIT) are used to achieve atomicity when writing data.

[0079] Version management: The model_version field is used to record the model iteration status (such as upgrading from "1.0" to "1.1"). The parent model reference field (such as "parentModel": "1.0") associates the extended model with the base model. Historical versions can be queried by model name and version number.

[0080] Interface provision: Provides a base model metadata query interface (such as GET / model / {model_id} / base) to the extended configuration module, returning the currently effective base model metadata (such as the attribute list, classification tree, and relationship diagram of the customer master data) as the initial basis for the extended configuration.

[0081] Extended configuration module: zero coding to achieve model extension The extension configuration module is the interactive core of the system. It supports business personnel to complete model extension without coding through a visual interface and generate extended model metadata. This module is implemented in the following ways: Visual Interface Interaction: Developed based on the Vue framework, the front-end includes three areas: a property list (displaying real-time attributes of the base model, such as "customerID" and "name"), a classification tree (displaying the classification system of the base model, such as "Customer Industry → Existing Manufacturing"), and a relationship diagram (displaying the relationships between the base model, such as "Customer-Supplier"). Business personnel can add new attributes by dragging and dropping (entering the name, data type, and constraint rules), add new categories from the right-click menu (recording versions and mapping rules), and add new relationships through the relationship diagram configuration (selecting objects and defining rules). For example, to add an "Environmental Certification Number" attribute to a product model, drag the "Add Property" component and enter "certificateNo," "string," and "maxLength: 20."

[0082] Metadata fragment generation: After the front-end component calls the enterprise terminology library interface to verify the compliance of the attribute name, it generates an attribute extension metadata fragment; the classification extension generates a fragment including the version and mapping rules; the association extension generates a fragment including the rules.

[0083] Metadata merging and storage: After the expansion is completed, the backend merging interface is called to merge each fragment with the basic model metadata, generate the extended model metadata in the "pending verification" state, and store it in the metadata database of the dynamic modeling module.

[0084] Adaptation engine module: Verify the business adaptability of the extended model The adaptation engine module is the verification core of the system, responsible for verifying the feasibility of the extended model in terms of technical implementation, data compatibility, and business compliance, and outputting the adaptation results. This module is implemented in the following ways: Interface adaptation: Call the Swagger Hub API gateway metadata service to obtain the real-time field definition of the current master data interface, parse the newly added attributes of the extended model (such as "certificateNo"), generate the interface field extension script and update the OpenAPI document, so that the business system can obtain the latest interface specifications.

[0085] Rule conflict detection: This function reads data quality rules (such as "customerIDUNIQUE") from the database table quality_rules and verifies whether the newly added attribute conflicts with existing rules (for example, whether "certificateNo UNIQUE" overlaps with "productCode UNIQUE"). A report is generated, including the conflicting rule ID and reason, and sent to the administrator console for review. If the conflict is resolved, the extended model metadata is marked as "effective."

[0086] Lifecycle management module: Synchronously extend the model to the business layer The lifecycle management module is the implementation unit of the system, responsible for synchronizing the effective extension model to the service layer, cleaning layer, and monitoring layer to achieve full lifecycle monitoring. This module is implemented in the following ways: Service layer synchronization: Publish the model_update topic message through the RabbitMQ message bus, including the expanded interface field definition (for example, the GET / customer return field adds "certificateNo"), notify the business system (such as ERP, CRM) to subscribe to the topic and trigger local interface adaptation (such as updating the API client's response parsing logic) to ensure that the business system can correctly receive the new data.

[0087] Cleaning layer synchronization: Call the Apache Spark data cleaning engine to read the attribute constraint rules of the effective extended model (such as "certificateNo length ≤ 20") from the dynamic modeling module, write the rules into the cleaning task configuration (such as "WHERELENGTH(certificateNo) ≤ 20"), and automatically apply the rules to newly connected or updated master data instances to filter or correct data that does not meet the constraints.

[0088] Monitoring layer synchronization: Extract extended attributes (such as "certificateNo") from the dynamic modeling module, add records (such as the indicator "certificateNo missing rate") to the monitoring indicator configuration table, and use scheduled tasks (such as 0:00 every day) to collect data and calculate the missing rate. The data is displayed on the Grafana dashboard, allowing business personnel to monitor data quality in real time.

[0089] Final status record: After synchronization of all layers is complete, the status of the extended model metadata in the dynamic modeling module is updated to "finally effective." The synchronization timestamp and the associated systems (such as the ERP procurement module and the CRM customer management module) are recorded, marking the official launch of the extended model. This enables full-process management of the master data model from demand capture to business implementation, resolving the issues of slow model expansion response and high manual intervention in existing methods, enabling enterprises to rapidly adapt to dynamic business changes.

[0090] The present invention solves the problem of missing extension benchmarks through a metadata-driven basic model construction mechanism. First, based on the enterprise's common master data objects (such as customers and products), a basic model structure including core attributes, classification systems, unique identification rules and association relationships is defined, and stored in a model warehouse that supports version management in the form of JSON format metadata descriptors. The metadata includes fields such as the model version number and the parent model reference, providing a traceable benchmark for subsequent extensions. For example, the basic model metadata of customer master data clearly states that "customer ID" is a required string and that "customer type" includes the categories of "individual customer" and "corporate customer". Subsequent extensions can directly overlay elements based on this benchmark, avoiding repeated definition of the model framework and shortening the expansion preparation time.

[0091] The present invention achieves rapid response to business changes through demand perception and zero-coding extension configuration. On the one hand, event monitoring components are deployed in the database change logs and API call logs of business systems such as ERP and CRM, and event contexts are extracted by matching keywords such as "new product line" and "adjust customer classification" through regular expressions; on the other hand, a Web form interface is opened to receive manual extension requirements, and both are uniformly converted into a structured requirement description including event type, target model and extension details through a message queue. Business personnel use the visual interface to drag and drop new attributes, right-click to expand categories, and configure associations through relationship diagrams to generate extended metadata fragments and merge them with the basic model. No code is required throughout the process, reducing the time required for manual model adjustment from hours / days to minutes, significantly improving response efficiency.

[0092] In this master data management solution, regular expressions serve as intelligent text recognition tools, automatically capturing key change signals within business systems. Their core value lies in converting unstructured system logs into machine-readable business events, eliminating the need for manual log sifting. The specific implementation logic is as follows: In business system log monitoring scenarios, the event monitoring component has multiple predefined regular expression rules built into it, continuously scanning the operation logs of systems such as ERP and CRM. For example, when a description such as "new product specifications" appears in an ERP database change log, the regular expression "new.*(product line|product specifications|category)" will immediately recognize the text pattern and automatically mark it as a "product model expansion request." When the CRM system's operation log records the "Customer Industry Type Adjustment" operation, the regular rule "Adjustment.*(Customer Type | Industry Classification)" extracts the operation object and change type to generate a structured event record.

[0093] During the demand conversion phase, the identified text information is intelligently decomposed using regular expressions. For example, from the text description of "Adding an environmental certification field to customer master data," the operation object (customer master data), operation type (adding a field), and field name (environmental certification) are automatically decomposed. From the request of "Adding a new energy industry subcategory for suppliers," the target object (supplier classification), operation instruction (adding a subcategory), and classification name (new energy) are accurately extracted.

[0094] The present invention implements the business implementation of the extended model through adaptation verification and full life cycle synchronization. After the extended model is generated, the system calls the API gateway metadata service to automatically expand the master data interface fields, combines the data quality rule library to detect rule conflicts, and marks it as effective after verification. The effective model is synchronized to the service layer (triggering business system interface adaptation), the cleaning layer (writing data cleaning rules) and the monitoring layer (adding new quality monitoring indicators) through the message bus, and each layer executes the extended implementation operation in parallel. For example, when the "environmental certification number" attribute is added to the product master data, the service layer interface synchronizes the new field, the cleaning layer automatically filters the overlong data, and the monitoring layer tracks the field missing rate in real time, realizing end-to-end closed-loop management of the extended model from definition to application, and realizing the rapid adaptation of the master data model when the enterprise business changes dynamically.

Claims

1. A master data management method based on enterprise data governance, characterized by: include: Step 1: Define the basic model structure based on the enterprise's common master data objects. The basic model structure includes core attributes, classification system, unique identification rules, and association relationships, and is stored in the form of metadata in a model repository that supports version management. Step 2: Obtain business system change events and manually submitted extension requirements, and convert the events and requirements into structured extension requirements that include event types, target master data models, and extension detail fields. Step 3: Based on the structured extension requirements and the basic model metadata stored in the model warehouse, complete the attribute, classification, and relationship extension configuration of the master data model through a visual interface to generate extended model metadata; Step 4: Perform interface adaptation, data mapping adaptation, and rule conflict detection on the extended model metadata. After verification, mark it as valid. Step 5: Synchronize the effective extended model metadata to the service layer, cleaning layer, and monitoring layer. The service layer is used for master data service interface management, the cleaning layer is used for data cleaning processing, and the monitoring layer is used for data quality monitoring. This is used for extended synchronization and monitoring throughout the entire life cycle of master data.

2. The master data management method based on enterprise data governance according to claim 1, characterized in that: Acquiring business system change events involves deploying event monitoring components for log collection in the business system's database change logs and API call logs. Regular expressions are used to match business change keywords and extract event context, including the change time, involved systems, and affected business objects. Obtaining manually submitted extension requirements includes receiving the extension object, new attribute name, and category adjustment range filled in by business personnel through the management console web form interface open to business personnel; The event context extracted by the event listening component and the manually submitted extension requirements are uniformly converted into structured extension requirements for the event type, target master data model, and extension detail fields in step 2 through the message queue.

3. The master data management method based on enterprise data governance according to claim 2, characterized in that: Also includes: Core attributes, classification system, unique identification rules and association relationships; The basic model structure is stored in the form of a metadata descriptor in JSON format. The metadata descriptor includes the model version number, the parent model reference, the type field that maps the basic model structure elements, and the constraint condition field corresponding to the unique identification rules and association relationships; The model warehouse stores basic model metadata including historical versions through database transactions.

4. The master data management method based on enterprise data governance according to claim 3, characterized in that: Also includes: Read the defined basic model metadata from the model repository, based on the real-time attribute list, classification system and association relationship of the basic model; Add attributes to the attribute list area by dragging and dropping, enter the attribute name, data type, and constraint rules, associate the enterprise terminology base, and generate attribute extended metadata fragments; In the classification system area, the real-time classification system of the basic model is displayed in a tree diagram. Users can right-click and select nodes to add sub-categories, record classification versions and mapping rules between old and new classifications, and generate classification extension metadata fragments. In the relationship area, use the relationship diagram component to display the existing relationships of the basic model, select new relationship objects, configure the relationship rules, and generate the relationship metadata fragment; After the extension is completed, the extended metadata fragments of attributes, classifications, and associations are merged with the basic model metadata to generate extended model metadata marked as pending verification.

5. The master data management method based on enterprise data governance according to claim 4, characterized in that: Also includes: Call the business system API gateway metadata service to obtain the field definition of the real-time master data interface, parse the newly added attributes in the extended model metadata, automatically generate the interface field extension script, and update the OpenAPI document of the master data service interface; Data lineage analysis tools display the topological relationship of master data; Read the data quality rule library, verify whether the newly added attributes of the extended model metadata conflict with existing rules, generate a report including the conflict rule ID and the cause of the conflict, and push it to the administrator console for review; If no conflict is detected or the conflict has been resolved, the extended model metadata is marked as valid.

6. The master data management method based on enterprise data governance according to claim 5, characterized in that: Also includes: Publish the expanded master data service interface field definition through the message bus, notify the business system to subscribe to the topic, and trigger the business system's local interface adaptation; The data cleaning engine reads the attribute constraint rules of the effective extended model metadata from the model warehouse, writes the rules into the cleaning task configuration, and automatically applies the rules to newly connected or updated master data instances; Extract the extended attributes of the effective extended model metadata from the model warehouse, add new records to the monitoring indicator configuration table, collect field data of the master data instance through scheduled tasks, calculate the missing rate and display it on the dashboard; after the synchronization of each layer is completed, record the final effective status of the extended model.

7. The master data management method based on enterprise data governance according to claim 6, characterized in that: Business systems include ERP and CRM systems. Event monitoring components are deployed in the ERP's MySQL Binlog log and the CRM's API call log. Regular expressions are used to match keywords related to business changes such as adding new product lines and adjusting customer categories, extracting event context including the change time, involved systems, and affected business objects. After the event context is parsed by the event listening component, it is pushed to the demand processing engine through the Kafka message queue. Manually submitted extension requirements are filled in through the web form interface of the management console. The form fields include the extension object, the name of the new attribute, and the category adjustment range. After submission, it is converted into a structured extension requirement description in JSON format and pushed to the demand processing engine through the same Kafka message queue.

8. The master data management method based on enterprise data governance according to claim 7, characterized in that: Core attributes are the basic fields that describe master data objects in the basic model structure, including customer ID and name; The classification system is a hierarchical division of master data objects in the basic model structure, including customer type and industry hierarchy; The unique identification rule defines the generation logic of the unique identification of the master data object in the basic model structure, including the customer ID generation algorithm; The element types of the metadata descriptor correspond to the components of the basic model structure, and the constraints include attribute mandatory and length-limited fields; The model warehouse uses MySQL database transactions to write basic model metadata, which is used to query historical versions or real-time effective versions by model name and version number.

9. The master data management method based on enterprise data governance according to claim 8, characterized in that: The API gateway is used to obtain the field definition of the real-time master data interface; The interface field extension script modifies the interface return fields and updates the OpenAPI document based on the newly added attributes of the extended model metadata; The data migration script extracts data from the table and writes it to the new table using a SELECT statement, applying the length constraint of the extended model metadata. The data quality rule base is stored in the database table, including rule types and rule expressions; The conflict report includes the conflict rule ID and the conflict reason, which is used by administrators to review the conflict resolution.

10. A master data management system based on enterprise data governance, applied to the master data management method based on enterprise data governance according to any one of claims 1 to 9, characterized in that: include: The demand perception module deploys event monitoring components in the business system log, parses change events and manual requirements into structured extended requirement descriptions, and pushes them to subsequent modules through the message queue; Dynamic modeling module, which stores basic model and extended model metadata, is used for version management and metadata query, and provides a basic model metadata interface to the extended configuration module; The extended configuration module reads the basic model metadata through a visual interface for drag-and-drop attribute expansion, tree diagram classification expansion, and relationship diagram association expansion, generates extended model metadata, and stores it in the dynamic modeling module; The adaptation engine module reads the extended model metadata, calls the business system API gateway and data quality rule library, performs interface adaptation, data mapping adaptation and rule conflict detection, and outputs the adaptation results; the lifecycle management module synchronizes the effective extended model metadata to the service layer, cleaning layer and monitoring layer. The service layer is used for master data service interface management, the cleaning layer is used for data cleaning processing, and the monitoring layer is used for data quality monitoring, and is used for extended synchronization and monitoring of the entire life cycle of master data.

Images