A master data management method and system based on enterprise data governance

Abstract

The present application relates to the technical field of data processing, and more particularly to a kind of master data management method and system based on enterprise data governance, including based on enterprise general master data object definition basic model structure and store in the form of metadata;Business system change event and artificial extension demand are obtained and converted into structured description;Attribute, classification and correlation are completed by visual interface Zero coding extension configuration;Interface adaptation, data mapping adaptation and rule conflict detection are executed;Synchronization of effective extension model to service layer, cleaning layer and monitoring layer.The system includes requirement perception, dynamic modeling, extension configuration, adaptation engine and life cycle management module.The present application realizes the whole process management of master data model from requirement capture to business landing, improves model extension efficiency and business adaptation capability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a master data management method and system based on enterprise data governance. BACKGROUND

[0002] The master data in enterprise data governance is a set of basic data entities shared by enterprises across business domains and with high business value, usually covering core business objects such as customers, suppliers, products, and employees, and is characterized by long-term relative stability, clear business attributes, and repeated reference in multiple systems or business processes. Master data management, as a key component of data governance, focuses on establishing an enterprise-level authoritative data source for master data through standardized definition, unified maintenance, and full-life-cycle control, providing consistent, accurate, and complete data benchmarks for cross-department business collaboration, data analysis and application, and system integration.

[0003] The master data management method based on enterprise data governance has the following pain points in terms of business adaptability. The existing method lacks the ability to expand and adjust the data model when the enterprise business changes dynamically, making it difficult to quickly respond to new business scenario requirements. For example, when an enterprise adds a product line due to market expansion, the existing master data management needs to manually redefine the characteristics, classification, attributes, and unique identifier of the master data, which is time-consuming and prone to human error, leading to lag in master data updates, affecting the timeliness of cross-system data sharing and the collaboration efficiency of business processes, and limiting the support of data for business innovation. SUMMARY

[0004] To address the deficiencies in the prior art, the present application provides a master data management method and system based on enterprise data governance, which solves the problem of insufficient data model expansion and adjustment capability in master data management, making it difficult to quickly respond to the dynamic changes of enterprise business requirements.

[0005] To solve the above technical problems, the specific technical solutions of the present application are as follows:

[0006] In a first aspect, the present application provides a master data management method based on enterprise data governance, comprising:

[0007] Step 1: Define a basic model structure based on enterprise general master data objects, which includes core attributes, classification system, unique identifier rules, and association relationships, and store it in a model repository supporting version management in the form of metadata as a benchmark for subsequent expansion;

[0008] Step 2: Obtain business system change events and manually submitted expansion requirements, and convert the events and requirements into structured expansion requirements including event type, target master data model, and expansion detail fields;

[0009] Step 3, according to the structured extension requirement, based on the basic model metadata stored in the model warehouse, the attribute, classification and correlation extension configuration of the master data model are completed through the visual interface, and the extension model metadata is generated;

[0010] Step 4, interface adaptation, data mapping adaptation and rule conflict detection are performed on the extension model metadata, and the verification is passed to mark as effective state;

[0011] Step 5, the effective extension model metadata is synchronized to the service layer, the cleaning layer and the monitoring layer, wherein the service layer is used for master data service interface management, the cleaning layer is used for data cleaning processing, and the monitoring layer is used for data quality monitoring, and is used for extension synchronization and monitoring of the whole life cycle of master data.

[0012] Further, the master data management method based on enterprise data governance of the application comprises the following steps:

[0013] The event listening component deployed for log collection in the database change log and API call log of the business system is matched with the business change keyword through a regular expression, and the event context including the change time, the involved system and the affected business object is extracted;

[0014] The event listening component extracts the event context and the manually submitted extension requirement, and uniformly converts them into the event type, the target master data model and the structured extension requirement of the extension detail field in step 2 through a message queue.

[0015] Further, the master data management method based on enterprise data governance of the application comprises the following steps:

[0016] The basic model structure is stored in the form of a JSON format metadata descriptor, and the metadata descriptor comprises a model version number, a parent model reference, a type field for mapping the basic model structure element, and a constraint condition field corresponding to the unique identification rule and the correlation.

[0017] The model warehouse stores the basic model metadata including historical versions through a database transaction.

[0018] Further, the master data management method based on enterprise data governance of the application further comprises the following steps:

[0019] The attribute is added in the attribute list area by a drag operation, an attribute name, a data type and a constraint rule are input, an enterprise term library is associated, and an attribute extension metadata fragment is generated;

[0020] In the classification system area, the real-time classification system of the basic model is displayed in a tree diagram, a node is selected through a right-click menu, and a sub-classification is added, a classification version and a new-old classification mapping rule are recorded, and a classification extension metadata fragment is generated;

[0021] In the association relationship area, the existing relationship of the basic model is displayed through a relationship diagram component, an associated object is selected, an association rule is configured, and an association relationship metadata fragment is generated.

[0022] After the extension is completed, the extension metadata fragments of the attributes, the classification and the association relationship are merged with the basic model metadata to generate an extended model metadata marked as a to-be-verified state.

[0023] Further, the enterprise data governance-based master data management method of the present application further comprises:

[0024] The business system API gateway metadata service is called to obtain the field definition of the real-time master data interface, the added attribute in the extended model metadata is parsed, the interface field extension script is automatically generated, and the OpenAPI document of the master data service interface is updated;

[0025] The data blood relationship analysis tool displays the master data topological relationship;

[0026] The data quality rule library is read, whether the added attribute of the extended model metadata conflicts with the existing rule is checked, a report including the conflict rule ID and the conflict reason is generated, and the report is pushed to the administrator console for auditing;

[0027] If no conflict is detected or the conflict has been solved, the extended model metadata is marked as an effective state.

[0028] Further, the enterprise data governance-based master data management method of the present application further comprises: publishing the extended master data service interface field definition through a message bus, notifying the business system subscription topic, and triggering the business system local interface adaptation;

[0029] The data cleaning engine reads the attribute constraint rule of the effective extended model metadata from the model warehouse, writes the rule into the cleaning task configuration, and automatically applies the rule to the newly accessed or updated master data instance;

[0030] The extended attribute of the effective extended model metadata is extracted from the model warehouse, a record is added in the monitoring index configuration table, the field data of the master data instance is collected through a timing task, the missing rate is calculated and displayed on the dashboard; after the synchronization of each layer is completed, the system records the final effective state of the extended model.

[0031] Further, the enterprise data governance-based master data management method of the present application, the business system includes an ERP, a CRM business system, an event listening component is deployed in the business system, and a regular expression is matched to add a product line and adjust a customer classification business change keyword, event context including a change time, a related system and an affected business object is extracted;

[0032] After the event context is parsed by the event listening component, it is pushed to the demand processing engine through the Kafka message queue; an extended demand submitted manually is filled in through a Web form interface of a management console, form fields include an extended object, an added attribute name and a classification adjustment range, and after submission, it is converted into a structured extended demand description in JSON format and pushed to the demand processing engine through the same Kafka message queue.

[0033] Further, the enterprise data governance-based master data management method of the present application, the core attribute is a basic field for describing a master data object in a basic model structure, including a customer ID and a name;

[0034] The classification system is a hierarchical division of the master data object in the basic model structure, including a customer type and an industry hierarchical structure;

[0035] The uniqueness identification rule is a generation logic for defining a unique identification of the master data object in the basic model structure, including a customer ID generation algorithm;

[0036] The element type of the metadata descriptor corresponds to a component part of the basic model structure, and the constraint condition includes an attribute mandatory and a length limit field;

[0037] The model warehouse realizes writing of the basic model metadata through a MySQL database transaction, and is used for querying a historical version or a real-time effective version according to a model name and a version number.

[0038] Further, the enterprise data governance-based master data management method of the present application, the API gateway is used for acquiring a field definition of a real-time master data interface;

[0039] The interface field extension script modifies an interface return field according to an added attribute of the extended model metadata, and updates an OpenAPI document;

[0040] The data blood relationship analysis tool displays a master data topological relationship;

[0041] The data migration script extracts from a table and writes into a new table through a SELECT statement, and applies a length constraint of the extended model metadata;

[0042] The data quality rule library is stored in a database table, including a rule type and a rule expression;

[0043] The conflict report includes a conflict rule ID and a conflict reason, and is used for an administrator to audit a conflict solution.

[0044] In a second aspect, the present application provides a master data management system based on enterprise data governance, which is applied to a master data management method based on enterprise data governance, and comprises:

[0045] A requirement perception module deploys an event listening component in a business system log, parses a change event and a manual requirement into a structured extended requirement description, and pushes the structured extended requirement description to a subsequent module through a message queue.

[0046] A dynamic modeling module stores basic model and extended model metadata, supports version management and metadata query, and provides a basic model metadata interface to an extended configuration module.

[0047] The extended configuration module reads the basic model metadata through a visual interface, supports drag-and-drop attribute extension, tree diagram classification extension and relationship diagram association extension, generates extended model metadata, and stores the extended model metadata in the dynamic modeling module.

[0048] An adaptation engine module reads the extended model metadata, calls a business system API gateway and a data quality rule library, performs interface adaptation, data mapping adaptation and rule conflict detection, and outputs an adaptation result. A life cycle management module synchronizes the effective extended model metadata to a service layer, a cleaning layer and a monitoring layer.

[0049] The present application has the following beneficial effects.

[0050] The present application provides a traceable benchmark for extension by storing the core attributes, classification system, unique identification rules and association relationships of enterprise general master data objects in a versioned model warehouse in the form of JSON metadata through a metadata-driven basic model construction mechanism, avoids repeated definition of model frameworks, and shortens the extension preparation time. The present application significantly improves the business response efficiency compared with the existing manual model adjustment by converting the business system change event and the manual extension requirement into a structured description through the requirement perception module, completing the attribute, classification and association relationship extension configuration through a visual interface without coding, and automatically extending the master data interface, generating a historical data migration script and detecting rule conflicts through the adaptation engine module. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, for those skilled in the art, other drawings can also be obtained from the drawings without any creative effort.

[0052] Figure 1 A flowchart of a master data management method based on enterprise data governance provided by the embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the objects, technical solutions and advantages of the present application clearer, the following will describe the technical solutions of the present application in detail with reference to the specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort fall within the scope of protection of the present application. The following will describe the technical solutions provided by the embodiments of the present application in detail with reference to the drawings. In order to better understand the objects of the present application, the following will further describe the present application in detail.

[0054] In the first aspect, referring to Figure 1 The master data management method based on enterprise data governance provided by the present application comprises the following steps.

[0055] Step 1: defining a basic model structure based on enterprise general master data objects, wherein the basic model structure comprises core attributes, classification system, unique identification rules and correlation relationship, and is stored in a model warehouse supporting version management in the form of metadata as a benchmark for subsequent extension;

[0056] Step 2: obtaining business system change events and manually submitted extension requirements, and converting the events and requirements into structured extension requirements including event type, target master data model and extension detail field;

[0057] Step 3: according to the structured extension requirements, based on the basic model metadata stored in the model warehouse, completing the extension configuration of the attributes, classification and correlation relationship of the master data model through a visual interface, and generating extension model metadata;

[0058] Step 4: performing interface adaptation, data mapping adaptation and rule conflict detection on the extension model metadata, and marking as effective state after verification;

[0059] Step 5: synchronizing the effective extension model metadata to a service layer, a cleaning layer and a monitoring layer, wherein the service layer is used for master data service interface management, the cleaning layer is used for data cleaning processing, and the monitoring layer is used for data quality monitoring, and is used for extension synchronization and monitoring in the whole life cycle of master data.

[0060] The application provides an enterprise data governance-based master data management method, which realizes dynamic expansion of a master data model through a phased metadata-driven mechanism, and the core steps and specific implementation modes are as follows:

[0061] First, define a basic model structure based on enterprise general master data objects. The enterprise general master data objects usually cover core entities such as customers, products, and suppliers that are shared across business domains, and the basic model structure needs to clearly define the core attributes of these entities (such as the basic fields of customer ID and name of the customer object), the classification system (such as the division of customer types into individual customers and enterprise customers, and the division of industries into existing manufacturing and new energy hierarchical structures), the uniqueness identification rule (such as the generation algorithm of "enterprise code + timestamp" for customer ID to achieve global uniqueness), and the association relationship (such as the cooperation relationship between customers and suppliers). The model structure is stored in a model warehouse supporting version management (such as metadata writing through MySQL database transactions) in the form of JSON metadata descriptors, which include model version number (such as the initial version "1.0"), parent model reference (marked as "null" when there is no parent model for the initial model), element type (attribute / classification / rule), and constraint conditions (such as customer ID as a required field, and name length ≤ 50) fields, providing a traceable benchmark for subsequent model expansion.

[0062] Second, obtain business system change events and manually submitted expansion requirements and convert them into structured descriptions. Business systems usually include ERP, CRM, and other core systems. Event listening components are deployed in the MySQL Binlog logs of ERP and the API call logs of CRM, and regular expression matching is used to match business change keywords such as "add new product line" and "adjust customer classification" to extract event context including change time, involved systems (such as ERP), and affected business objects (such as products). At the same time, a Web form interface of the management console is opened to receive the expansion objects (such as customer models) filled by business personnel, new attribute names (such as "product environmental protection certification number"), and classification adjustment ranges (such as "add new energy classification to customer industry"). The system converts the parsed system event context and manual requirements into a JSON-formatted structured expansion requirement description through a Kafka message queue, which includes eventType (business change / manual application), targetModel (target master data model such as "customer model"), and extensionDetails (extension details such as a list of new attributes), and serves as the input basis for model expansion.

[0063] Third, the extension configuration of the master data model is completed according to the structured extension requirements. The system reads the basic model metadata stored in the model repository through the visual interface, and based on the real-time attribute list, classification tree and relationship diagram of the basic model, performs extension operations in three categories: in the attribute list area, business personnel add attributes (such as "product environmental protection certification number") through drag-and-drop operations, input attribute names, data types (such as strings) and constraint rules (such as length ≤ 20), associate enterprise terminology library (such as through the terminology ID to associate "environmental protection certification number") to generate attribute extension metadata fragments; in the classification system area, the current classification of the basic model is displayed in a tree diagram (such as "customer industry → existing manufacturing"), a node is selected through the right-click menu and a "new sub-classification" is added (such as "new energy"), the classification version is recorded (such as upgrading from V1.0 to V1.1) and the new and old mapping rules are recorded (such as "old classification 'existing manufacturing' is retained, and 'new energy' is added") to generate classification extension metadata fragments; in the associated relationship area, the existing relationship is displayed through the relationship diagram component (such as "customer-supplier"), a new associated object is selected (such as "product" and "research and development team") and rules are configured (such as "the product belongs to the research and development team ID must exist in the research and development team master data") to generate associated relationship extension metadata fragments. After the extension is completed, the system combines each type of extension metadata fragment with the basic model metadata to generate an extension model metadata marked as "to be verified".

[0064] Fourth, interface adaptation, data mapping adaptation and rule conflict detection are performed on the extended model metadata to verify its validity. In the interface adaptation stage, the system calls the API gateway metadata service to obtain the field definition of the current master data interface (such as the GET / customer interface returns the fields customerID and name), parses the added attributes of the extended model metadata (such as "certificateNo"), automatically generates an interface field extension script (such as modifying the interface return fields to customerID, name, and certificateNo), and updates the OpenAPI document. In the data mapping adaptation stage, the storage location of the historical master data instance (such as the Hive table customer_v1) is tracked through data lineage analysis tools such as Apache Atlas, the historical fields related to the extended attributes (such as "authentication notes") are extracted, and the data migration script is generated based on the attribute constraints of the extended model (such as the length of certificateNo is ≤20) (such as writing the first 20 bits of the remark field from the customer_v1 table to the new table customer_v2). In the rule conflict detection stage, the data quality rule library (stored in the database table quality_rules, including the rules "customerID UNIQUE" and the like) is read, the added attributes are checked for conflicts with existing rules (such as if "certificateNo UNIQUE" is added, it needs to be checked for duplication with "productCode UNIQUE"), and a report including the conflict rule ID and reason (such as {"ruleID":"R001","reason":"certificateNo and productCode uniqueness rules are duplicated"} is generated and pushed to the console for review. If no conflict is detected or the conflict has been resolved, the extended model metadata is marked as "effective" state.

[0065] Finally, the effective extended model metadata is synchronized to the service layer, the cleansing layer and the monitoring layer to realize the whole life cycle management. The service layer publishes the extended interface field definition (such as the customer interface including certificateNo) through a message bus such as RabbitMQ, notifies the business system to subscribe to the model_update topic and triggers the local interface adaptation (such as updating the interface return field); the cleansing layer reads the attribute constraint rules (such as the length of certificateNo≤20) of the extended model through a data cleansing engine such as Apache Spark, writes the rules into the cleansing task configuration (such as WHERE LENGTH(certificateNo)<=20), and the rules are automatically applied to the newly accessed or updated master data instances; the monitoring layer extracts the extended attribute (such as certificateNo) from the model repository, adds the "certificateNo missing rate" index (such as metricID is M005) to the monitoring index configuration table, collects the certificateNo field data of the master data instance through a timing task (such as every day at 0 o'clock), calculates the missing rate and displays it on the dashboard. After the synchronization of each layer is completed, the system records the final effective state of the extended model, and forms a complete closed loop from demand perception to extension landing.

[0066] The above steps are connected through a metadata-driven process, realizing the whole process management of the master data model from basic definition to dynamic extension, and improving the sharing efficiency and business support ability of the master data.

[0067] Specifically, the master data management method based on enterprise data governance of the application acquires a business system change event, includes deploying an event listening component for log collection in a database change log and an API call log of the business system, matching a business change keyword through a regular expression, and extracting an event context including a change time, a related system and an affected business object;

[0068] The application acquires an artificial submitted extension requirement, includes receiving an extension object, a new attribute name and a classification adjustment range filled by a business personnel through an open management console Web form interface operated by the business personnel;

[0069] The event context extracted by the event listening component and the artificial submitted extension requirement are uniformly converted into an event type, a target master data model and a structured extension requirement of an extension detail field in step 2 through a message queue.

[0070] In the business system change event capture link, the event listening component deployed in the database change log and API call log runs as a lightweight agent program. The component preinstalls a regular expression matching rule library and continuously scans the log content; when a log entry including the predefined business change keywords such as "new product line" and "customer classification adjustment" is detected, the triple information is automatically extracted: the precise timestamp of the change, the source business system module name involved, and the business object type affected. For example, when a table structure change statement is identified in the MySQL Binlog log of the ERP system, the event listening component marks the operation time as the event triggering time, records the system module as "purchase management", and the business object as "product master data".

[0071] The artificial extension requirement is collected through an interactive Web form interface. The management console provides a tree selector for the business personnel to select the target master data model, an input box receives the new attribute name and associates the enterprise terminology library to automatically check the naming specification, and the classification adjustment range is specified by a hierarchical selector and the new sub-class name is input. After the form is submitted, the backend service verifies the logical integrity of the input data, including attribute data type compliance and classification hierarchy validity, and converts the artificial operation intention into a standard data format.

[0072] The event context and the artificial requirement are unified in format through a distributed message middleware. The context triple output by the event listening component and the extension parameters submitted by the artificial are packaged into a JSON structure and sent to the message queue through a Kafka producer; the structure includes an event type identifier, a target master data model name, and an extension operation detail list. The requirement processing engine subscribes to the message topic as a consumer and uniformly parses the heterogeneous source input into a structured extension requirement object including an event type field, a model identifier field, and an extension field list.

[0073] The logical connection relationship of each step is: the event listening component realizes real-time capture of business system change signals, the management console provides a standardized input channel for artificial requirements, the message queue mechanism solves the transmission problem of heterogeneous data sources, and finally outputs a standardized requirement description for subsequent extension configuration module calling. The technical terms are kept unified, "event listening component" refers to the log scanning agent, "management console" refers to the Web interactive interface, and "structured extension requirement" refers to the JSON data object including fixed fields.

[0074] Specifically, the master data management method based on enterprise data governance of the application has a basic model structure based on a model framework defined based on enterprise general master data objects, and specifically includes core attributes, a classification system, a uniqueness identification rule, and a correlation relationship.

[0075] The base model structure is stored in the form of metadata descriptors in JSON format, and the metadata descriptors include a model version number, a parent model reference, a type field mapping the base model structure elements, and a constraint condition field corresponding to the uniqueness identification rule and the association relationship;

[0076] The model repository stores the base model metadata including historical versions through database transactions.

[0077] The definition, storage and management of the base model structure in the application are the cornerstone of the dynamic expansion of master data, and the specific implementation and logical relationship are as follows:

[0078] The base model structure is based on enterprise general master data object definition and covers core entities shared across business domains. The enterprise general master data object usually includes frequently used business entities such as customers, products and suppliers, for example, the customer master data object is used to describe the interaction relationship between the enterprise and the external individual or organization, and the product master data object is used to define the attributes of goods or services provided by the enterprise. The base model structure needs to clearly define the core elements of these entities: core attributes are basic fields for describing the essential characteristics of the object, such as "customer ID" (unique identification field) and "name" (business identification field) of the customer object; classification system is the hierarchical division of the object, such as "personal customer" and "enterprise customer" for customer type, and "existing manufacturing" and "new energy" for industry; the uniqueness identification rule is the generation logic for realizing the global uniqueness of the object, such as the combination algorithm of "enterprise code (6 digits) + timestamp (14 digits)" for customer ID; the association relationship is the business connection rule between objects, such as the "customer-supplier" association defined as "the main supplier ID of the customer must exist in the supplier master data".

[0079] The basic model structure is stored in the form of metadata descriptor in JSON format, realizing machine reading and standardization of model elements. The metadata descriptor includes the following core fields: model version number (such as the initial version "1.0", which is upgraded to "1.1" after extension), used to identify the iteration state of the model; parent model reference (the initial model is "null", and the extended model references the basic model version number "1.0"), used to trace the inheritance relationship of the model; element type (distinguish "attribute" attribute, "category" category, "rule" rule), to clarify the category of the model element; constraint condition (such as "required: true" indicating that the attribute is required, "maxLength: 50" indicating that the attribute length is limited), to define the business rules of the model element. The model repository realizes the storage and management of the basic model metadata through database transactions, providing a traceable benchmark for subsequent extension. The model repository uses MySQL database, and realizes the atomicity of metadata writing through transaction operation (such as BEGIN...COMMIT), to avoid data inconsistency. The repository supports querying historical versions or current effective versions according to the model name (such as "customer_master") and version number (such as "1.0"), for example, obtaining the basic model metadata of customer master data. The version management function allows developers to view the iteration records of the model (such as the change difference from "1.0" to "1.1"), and the metadata traceability function associates the extended model with the basic model through the parent model reference field (such as "parentModel: '1.0'"), realizing the auditability of the extension operation.

[0080] The above design builds a benchmark framework for master data model extension through element definition of specific entities, standardized metadata description and versioned warehouse management, so that subsequent model extension can be based on clear metadata basis, ensuring the standardization and traceability of the extension operation.

[0081] Specifically, the master data management method based on enterprise data governance of the application further includes: reading the defined basic model metadata from the model repository, and based on the real-time attribute list, classification system and association relationship of the basic model, including:

[0082] An attribute is added in the attribute list area through a drag-and-drop operation, an attribute name, a data type and a constraint rule are input, an enterprise terminology library is associated, and an attribute extension metadata segment is generated;

[0083] The real-time classification system of the basic model is displayed in a tree diagram in the classification system area, a node is selected and a sub-classification is added through a right-click menu, a classification version and a new-old classification mapping rule are recorded, and a classification extension metadata segment is generated;

[0084] The existing relationship of the basic model is displayed through a relationship diagram component in the association relationship area, a new association object is selected, an association rule is configured, and an association relationship metadata segment is generated;

[0085] After the extension is completed, the extended metadata segments of the attributes, categories, and association relationships are merged with the basic model metadata to generate an extended model metadata marked as a to-be-verified state.

[0086] The visual interface extension configuration in the application is a core interactive link for realizing zero coding extension of the main data model, and the specific implementation manner and logical relationship are as follows:

[0087] The visual interface first reads the basic model metadata from the model warehouse as the initial basis for extension configuration. The system obtains the real-time attribute list, category system, and association relationship data of the basic model by calling the metadata query interface (such as GET / model / {model_id} / base) of the model warehouse. For example, for the customer main data model, the read attribute list can include fields such as "customerID" and "name", the category system is displayed in a tree structure as "customer type -> personal customer / enterprise customer" and "industry -> existing manufacturing / new energy", and the association relationship is displayed in the form of a relationship diagram as "customer-supplier" and "customer-product". These data are rendered into a visual interface by a front-end component (such as a React TreeView and GraphVis) for intuitive operation by business personnel.

[0088] Attribute extension is completed through a drag-and-drop operation in the attribute list area. The interface provides a "new attribute" component library on the left side, and business personnel drags the component to the attribute list area, and then an input box is popped up to require filling in the attribute name (such as "environmental protection certification number"), data type (select "string", "integer", etc. through a drop-down menu), and constraint rule (such as "length <= 20" and "required"). The system automatically calls an enterprise terminology library interface (such as GET / term / {term_id}) to verify the standardization of the attribute name (such as whether it is a defined standard term), and generates an attribute extension metadata segment after verification.

[0089] The extension of the classification system is realized through a tree interface. The interface displays the real-time classification system of the basic model in the form of a tree (e.g., "customer industry → existing manufacturing"). After the business personnel right-clicks the target node (e.g., "existing manufacturing"), they select the "add sub-category" option and input the new category name (e.g., "new energy"). The system automatically records the classification version change (e.g., from V1.0 to V1.1) and generates the mapping rules between the new and old categories (e.g., "old category 'existing manufacturing' remains, and 'new energy' is added as its sub-node"). This forms the classification expansion metadata fragment {"type":"category","name":"new energy","parent":"existing manufacturing","version":"1.1"}. For example, when a "new energy" sub-category is added in the customer industry classification, the system synchronously updates the classification tree structure and generates the corresponding classification expansion fragment.

[0090] The extension of the association relationship is completed through the configuration of a relationship graph component. The interface displays the existing associations of the basic model in the form of a relationship graph (e.g., the node "customer" is connected to the node "supplier" through the "cooperation relationship"). The business personnel drag the new association object (e.g., "product" and "R&D team") from the object library to the relationship graph area, select the connection edge, and configure the association rules (e.g., "the product's R&D team ID must exist in the R&D team master data"). The system verifies the validity of the rules (e.g., checks whether "R&D team ID" is a valid field in the R&D team master data) and generates the association relationship expansion metadata fragment. For example, when a new association between the product master data model and the R&D team is added, the system generates the corresponding association expansion fragment.

[0091] After the extension is completed, the system merges the expansion metadata fragments of attributes, classifications, and association relationships with the basic model metadata. The front-end component calls the back-end merge interface (e.g., POST / model / {model_id} / merge) to add each fragment to the "elements" list of the basic model metadata, generating the complete expansion model metadata. At the same time, the system updates the state field of the model in the metadata database to "to be verified" and records the timestamp and operator ID of the expansion operation (e.g., "operator: user001, timestamp: 2025-05-27 15:30:00"). For example, after the customer master data model completes the attribute, classification, and association expansion, the generated expansion model metadata includes the original elements of the basic model and the newly added expansion fragments, and the status is marked as "to be verified", waiting for subsequent interface adaptation and rule detection.

[0092] The above steps convert the expansion requirements of business personnel into structured metadata fragments through the interactive operation of the visual interface, finally generate a verifiable expansion model metadata by merging, realize zero coding expansion of the master data model, reduce the technical threshold of model adjustment, and improve the response efficiency of business requirements.

[0093] Specifically, the enterprise data governance-based master data management method of the application further comprises:

[0094] The business system API gateway metadata service is called to obtain the field definition of the real-time master data interface, the added attribute in the expansion model metadata is parsed, the interface field expansion script is automatically generated, and the OpenAPI document of the master data service interface is updated;

[0095] The storage location of the historical master data instance is tracked through a data blood relationship analysis tool, the historical field related to the expansion attribute is extracted, the data migration script is generated based on the attribute constraint of the expansion model metadata;

[0096] The data quality rule library is read, whether the added attribute of the expansion model metadata conflicts with the existing rule is checked, a report including the conflict rule ID and the conflict reason is generated, and the report is pushed to the administrator console for auditing;

[0097] If no conflict is detected or the conflict has been solved, the expansion model metadata is marked as an effective state.

[0098] In the application, interface adaptation, data mapping adaptation and rule conflict detection are key links for verifying the business adaptability of the expansion model metadata, and the specific implementation manners and logical relationships are as follows:

[0099] In the interface adaptation stage, the API gateway metadata service is called to realize the automatic expansion of the master data interface. The Swagger Hub and other API gateway metadata services are called by the system to obtain the real-time field definition of the current master data interface (for example, the return field of the GET / customer interface is ["customerID", "name"]). After the added attribute in the expansion model metadata is parsed (for example, the "certificateNo" attribute is extracted through the JSON field), the system automatically generates an interface field expansion script, for example, the return field of the interface is modified to ["customerID", "name", "certificateNo"]. After the script is executed, the system synchronously updates the OpenAPI document (for example, the "responses" part in the Swagger document) of the master data service interface, so that the business system can obtain the latest interface specification through the document. For example, when the "environmental protection certification number" attribute is added to the customer master data interface, the interface adaptation operation expands the original interface return field, so that the ERP and CRM systems calling the interface can obtain complete data including the new attribute.

[0100] The data mapping adaptation phase realizes the compatibility of historical data and the extended model through a data lineage analysis tool. The system calls data lineage analysis tools such as Apache Atlas to track the storage location of historical master data instances (such as the customer_v1 table in the Hive data warehouse) and extract historical fields related to extended attributes (such as the "authentication notes" field in the customer_v1 table). Based on the attribute constraints defined in the extended model metadata (such as "certificateNo" length ≤ 20), the system generates data migration scripts, such as the SQL statement "INSERT INTO customer_v2 (customerID, name, certificateNo) SELECT customerID, name, SUBSTR(remark, 1, 20) FROM customer_v1", to map the first 20 characters of the "authentication notes" field of historical data to the new attribute "certificateNo". This operation realizes the compatibility of historical data and the extended model structure, avoiding data gaps caused by model expansion.

[0101] The rule conflict detection phase verifies the compliance of the extended model through a data quality rule library. The system reads the data quality rules stored in the database table quality_rules (such as "customerID UNIQUE" "name NOT NULL"), and checks one by one whether the new attributes of the extended model conflict with existing rules. For example, if a new attribute is defined as "certificateNo UNIQUE", the system needs to check whether there are other fields (such as "productCode") in the quality_rules table that have defined the same type of uniqueness rule. If a conflict is detected (such as the "productCode UNIQUE" and "certificateNo UNIQUE" rules are repeated), the system generates a report including the conflict rule ID (such as "R001") and the conflict reason (such as "certificateNo and productCode uniqueness rules are repeated"), and pushes it to the administrator console. The administrator can adjust the constraint rules of the extended model (such as canceling the "certificateNo UNIQUE" definition) or modify the existing rules (such as adjusting the scope of "productCode UNIQUE") according to the report.

[0102] If no conflict is detected or the conflict has been resolved by manual adjustment, the system will update the state field of the model metadata from "to be verified" to "effective". The metadata in the effective state will be synchronized as the basis for subsequent service layer, cleaning layer and monitoring layer, realizing that the extended master data model is formally put into business use.

[0103] The above steps verify the feasibility of the extended model metadata in terms of technical implementation, data compatibility and business compliance through the ordered execution of interface extension, data migration and rule verification, and provide multi-dimensional protection for the formal effectiveness of the model.

[0104] Specifically, the master data management method based on enterprise data governance of the application further includes: publishing the extended master data service interface field definition through the message bus, notifying the business system to subscribe to the topic, and triggering the local interface adaptation of the business system;

[0105] The data cleaning engine reads the attribute constraint rules of the effective extended model metadata from the model warehouse, writes the rules into the cleaning task configuration, and automatically applies the rules to the newly accessed or updated master data instances;

[0106] The extended attributes of the effective extended model metadata are extracted from the model warehouse, and a new record is added to the monitoring index configuration table. The field data of the master data instance is collected through a timing task, the missing rate is calculated and displayed on the dashboard; after the synchronization of each layer is completed, the system records the final effective state of the extended model.

[0107] The synchronization of the service layer, the cleaning layer and the monitoring layer in the application is a key link for the landing application of the effective extended model metadata. Each layer performs dynamic management for the whole life cycle of the master data in parallel, and the specific implementation and logical relationship are as follows:

[0108] The service layer synchronizes the extended interface definition through the message bus to trigger business system adaptation. Based on the effective extended model metadata (e.g., customer master data model including a new attribute "certificateNo"), the system publishes a model_update topic message through a message bus such as RabbitMQ, and the message content includes the extended master data service interface field definition (e.g., the return field of the GET / customer interface is extended from ["customerID", "name"] to ["customerID", "name", "certificateNo"]). The business system (e.g., ERP, CRM) subscribes to the topic to obtain interface change notifications and calls local interface adaptation scripts (e.g., modifies the response parsing logic of the API client) to implement the ability to correctly receive and process new fields. For example, after the ERP system subscribes to the model_update topic, it detects the addition of the "certificateNo" field in the customer interface and automatically updates the customer data display interface of its procurement module to add a display column for the field.

[0109] The cleansing layer synchronizes the extended attribute constraint rules through the data cleansing engine to ensure data quality. The system calls a data cleansing engine such as Apache Spark to read the attribute constraint rules (e.g., "certificateNo length ≤ 20") in the effective extended model metadata from the model repository, converts the rules into filtering conditions (e.g., WHERE LENGTH(certificateNo)≤20) for cleansing tasks, and writes them into task configuration files. Subsequently, newly accessed master data instances (e.g., customer data imported from business systems through ETL tools) or updated instances (e.g., data submitted when a customer modifies environmental certification information) will automatically apply the rules before entering the master data warehouse, filtering or correcting data that does not meet the constraints (e.g., truncating the "certificateNo" field with a length of more than 20 to the first 20 characters). For example, a customer submits a "certificateNo" value of "CN-20250527-ABCDEFGHIJKLMN" (length 24), and the cleansing task will automatically truncate it to "CN-20250527-ABCDEFGHIJKL" (length 20), ensuring that the data meets the model constraints.

[0110] The monitoring layer synchronizes the data quality tracking of the extended attributes by adding monitoring indicators. The system extracts the extended attributes (such as "certificateNo") from the effective extended model metadata in the model repository, adds a record in the monitoring indicator configuration table (such as monitoring_metrics), including the indicator name (such as "certificateNo missing rate") and the display dimension (such as by customer type, industry classification). Through a timing task (such as triggered every day at 0 o'clock), the "certificateNo" field data of the master data instance is collected, the missing rate is calculated and stored in the monitoring database (such as InfluxDB). The monitoring dashboard (such as Grafana) queries the monitoring database to display the "certificateNo missing rate" trend chart in real time, and business personnel can evaluate the filling quality of the extended attribute (such as the missing rate is 15% in a week, prompting to strengthen data entry specification) through the indicator.

[0111] After each layer is synchronized, the system updates the state field of the extended model metadata in the model repository to "final effective", and records the synchronization completion timestamp (such as "2025-05-27 16:00:00") and the involved business systems (such as "ERP procurement module, CRM customer management module"). The status update indicates that the extended model has been fully integrated into the enterprise master data management process, and subsequent business operations will be based on the model for data interaction, cleaning and monitoring.

[0112] The above steps realize the synchronous landing of the extended master data model in the technical implementation, data quality and business monitoring levels through the parallel execution of service layer interface adaptation, cleaning layer rule application and monitoring layer indicator tracking, forming a complete closed loop from model extension to business application.

[0113] Specifically, the master data management method based on enterprise data governance of the application, the business system includes an ERP and a CRM business system, an event listening component is deployed in a MySQL Binlog log of the ERP and an API call log of the CRM, a regular expression is matched to add a product line and adjust a customer classification business change keyword, and an event context including a change time, an involved system and an affected business object is extracted;

[0114] After the event context is parsed by the event listening component, it is pushed to a demand processing engine through a Kafka message queue; an artificial submitted extension demand is filled through a Web form interface of a management console, form fields include an extension object, an added attribute name and a classification adjustment range, after submission, a Spring Boot backend service is converted into a JSON format structured extension demand description, and is pushed to the demand processing engine through the same Kafka message queue.

[0115] The acquisition and integration of the business system change event and the artificial expansion demand in the application are input links for realizing the dynamic expansion of the master data model, and the specific implementation manners and logical relations are as follows:

[0116] The capture of the business system change event is realized by deploying an event listening component in the core business system log. The enterprise core business system usually includes an ERP (Enterprise Resource Planning System) and a CRM (Customer Relationship Management System), and the event listening component needs to be deployed in the log sources of the two types of systems. For the ERP system, the database change log (such as the MySQL Binlog log) is selected, which records detailed information of all table structure changes (such as the ALTER TABLE operation) and data changes (such as the INSERT / UPDATE statement). For the CRM system, the API call log (such as the REST API access log based on Spring Boot) is selected, which records the request parameters and timestamps of the business operations (such as customer classification modification and supplier information update) triggered by the business personnel through the front-end interface. For example, when the ERP system performs the table structure change of "ALTER TABLE product ADD COLUMN new_feature VARCHAR(50)", the MySQL Binlog log will record the time (such as 2025-05-20 14:30:00), the table name (product) and the added field (new_feature) of the operation, and the event listening component captures the change event by listening to the log source.

[0117] The event listener component extracts business change keywords and contextual information through regular expression matching. The component predefines regular expression patterns to match business change keywords related to master data model extensions: for ERP database change logs, it matches keywords such as "ADD COLUMN" and "ALTER TABLE" to extract the change time (e.g., the "timestamp" field in the log), the system involved (marked as "ERP"), and the affected object (e.g., the "product" master data model); for CRM API call logs, it matches interface parameter keywords such as "update_category" and "add_attribute" to extract the change time (e.g., the "request_time" field in the request), the system involved (marked as "CRM"), and the affected object (e.g., the "customer" master data model). For example, when the CRM system API call log shows a request "POST / api / customer / update?action=add_category&category=new energy", the event listener component extracts the event context as "Change time: 2025-05-20 15:00:00, system involved: CRM customer management module, affected object: customer master data model" by matching the keyword "add_category".

[0118] The parsed event context is pushed to the demand processing engine via a Kafka message queue. The event listening component encapsulates the extracted system event context (such as an ERP table structure change event) into JSON format (e.g., {"eventType":"system","source":"ERP","targetModel":"product","details":"ADD COLUMN new_feature"}) and sends it to the "model_extension_request" topic via a Kafka producer. The demand processing engine (e.g., a Spring Kafka-based consumer) subscribes to this topic, receives and parses system event messages in real time, and uses them as one of the inputs for model extension.

[0119] Manually submitted extension requests are collected and transformed via a web form interface in the management console. Business users access the management console of the master data management system through a browser and enter the "Model Extension" page. This page provides a visual form with the following fields: Extension Object (select master data models such as "customer," "product," and "supplier" via dropdown menus), New Attribute Name (must conform to the enterprise terminology library specifications, such as "Environmental Certification Number"), and Category Adjustment Scope (select the parent category in a tree structure and enter the sub-category name, such as "Customer Industry → Existing Manufacturing → New Energy"). After the form is submitted, the Spring Boot backend service validates the input: attribute names must be verified as standard terms via a terminology library interface (e.g., GET / term / validate?name=Environmental Certification Number), and category adjustments must be checked to see if the parent category exists (e.g., whether "Existing Manufacturing" is an existing category for the customer industry). After successful validation, the backend converts the manual request into JSON format and sends it to the "model_extension_request" topic through the same Kafka producer, merging it with system event messages.

[0120] The demand processing engine receives and integrates system events and manual demand messages, triggering subsequent model extension processes in a unified manner. Through integration via Kafka message queues, system events and manual demands are transmitted to the demand processing engine in the same format (JSON) and with the same topic (model_extension_request). The engine distinguishes the message source based on the eventType field, extracts the targetModel (target model) and details (extension details) fields, and calls the model extension configuration module for subsequent operations.

[0121] The above steps, through log monitoring, keyword matching, message queue integration, and manual requirement standardization, transform scattered business change signals into machine-readable structured requirements, providing a clear and unified input basis for the subsequent dynamic expansion of the master data model.

[0122] Specifically, the master data management method based on enterprise data governance of the present invention has core attributes in the basic fields describing master data objects in the basic model structure, including customer ID and name;

[0123] The classification system is the hierarchical division of master data objects in the basic model structure, including customer type and industry hierarchy;

[0124] The unique identification rules define the generation logic for the unique identifier of master data objects in the basic model structure, including the customer ID generation algorithm;

[0125] The element type of the metadata descriptor corresponds to the component of the basic model structure, and the constraints include attribute mandatory and length limit field;

[0126] The model warehouse realizes the writing of the basic model metadata through a MySQL database transaction, and is used for querying historical versions or real-time effective versions according to the model name and version number.

[0127] The core component and storage management mechanism of the basic model structure in the application are the bottom support of the expansion of the master data model, and the specific implementation and logical relationship are as follows:

[0128] The core attribute is the basic description field of the master data object, and clearly defines the essential characteristics of the object. Taking the common customer and product master data objects of an enterprise as examples: the core attributes of the customer master data include "customer ID" (a globally unique identification field, such as "CUST-20250527-001") and "name" (a business identification field of the customer, such as "XX Technology Co., Ltd."); the core attributes of the product master data include "product ID" (such as "PROD-20250527-001") and "product name" (such as "new energy automobile battery"). These attributes are the basic identification of the master data object in cross-system interaction, and realize the consistency of identifying the same object in different business scenarios. For example, the ERP system associates the purchase order through the "customer ID", and the CRM system associates the customer service record through the "customer ID", and the two realize data interconnection through the core attribute.

[0129] The classification system realizes the business classification of the master data object through hierarchical division. Taking the customer master data as an example, the classification system includes two dimensions of "customer type" and "industry": "customer type" is a first-level classification, which is divided into "individual customer" and "enterprise customer", and "enterprise customer" can be further divided into "large enterprise" and "small and medium-sized enterprise" (second-level classification); "industry" is another first-level classification, which is divided into "existing manufacturing", "new energy", "information technology", etc., and "existing manufacturing" can be further divided into "automobile manufacturing" and "mechanical processing" (second-level classification). This hierarchical structure supports business personnel to filter and analyze data according to different dimensions, for example, the marketing department can quickly locate the target customer group through "customer type = enterprise customer" and "industry = new energy".

[0130] The uniqueness identification rule realizes the global uniqueness of the master data object through explicit generation logic. Taking the generation algorithm of customer ID as an example, the combination rule of "enterprise code (6 bits) + timestamp (14 bits) + sequence number (3 bits)" is adopted: the enterprise code is allocated when the enterprise is registered (such as "ABC001"), the timestamp is the current time accurate to milliseconds (such as "20250527153045000"), and the sequence number is used to process multiple object generations in the same millisecond (such as "001"). This algorithm realizes that each customer ID is unique within the enterprise, avoiding data confusion caused by repeated IDs. For example, when two customers are created in the same millisecond, the sequence numbers are "001" and "002" respectively, and the generated customer IDs are "ABC00120250527153045000001" and "ABC00120250527153045000002" respectively, realizing uniqueness.

[0131] The metadata descriptor standardizes the constituent elements of the field mapping base model and defines business rules. The "element type" field of the metadata descriptor corresponds to the constituent parts of the base model one by one: the "attribute" type corresponds to the core attributes (such as "customerID" "name"), the "category" type corresponds to the classification system (such as "customer type" "industry"), and the "rule" type corresponds to the uniqueness identification rule (such as "customer ID generation algorithm"). The constraint condition field defines the business rules of the attributes, for example, the constraint of "customerID" is {"required": true} (required), and the constraint of "name" is {"maxLength": 50} (length ≤ 50). For example, the metadata descriptor fragment of the customer master data can be represented as:

[0132] The model repository realizes reliable storage and query of metadata through MySQL database transactions. The metadata table (such as model_metadata) of the model repository includes fields such as "model_name" (model name, such as "customer_master"), "model_version" (model version, such as "1.0"), and "metadata_json" (metadata descriptor JSON string). When data is written, database transactions (BEGIN...COMMIT) are used to realize atomicity, for example:

[0133] Query supports retrieval by model name and version number, for example, get the base model metadata of customer master data through "SELECT metadata_json FROM model_metadata WHERE model_name='customer_master' AND model_version='1.0'". The version management function allows tracking the iteration history of the model (such as change records from "1.0" to "1.1"), and the metadata traceability function associates the extended model with the base model through the parent model reference field (such as "parentModel": "1.0"), achieving the auditable nature of the extension operation.

[0134] The above design builds a benchmark framework for the extension of master data models by explicitly identifying core attributes, hierarchically managing classification systems, algorithmically defining uniqueness rules, standardizing metadata descriptions, and versioning repository storage, providing a specification and traceable basis for subsequent dynamic adjustment of models.

[0135] Specifically, the master data management method based on enterprise data governance of the application comprises the following steps:

[0136] The interface field extension script modifies the interface return field according to the new attribute of the extended model metadata, and updates the OpenAPI document;

[0137] The data lineage analysis tool displays the master data topology relationship;

[0138] The data migration script extracts from the table through a SELECT statement and writes to a new table, and applies the length constraint of the extended model metadata;

[0139] The data quality rule library is stored in a database table, including rule types and rule expressions;

[0140] The conflict report includes conflict rule ID and conflict reason, and is used for administrator to review the conflict solution.

[0141] The collaborative application of the API gateway metadata service, data lineage analysis tool and data quality rule library in the application is a technical support link for verifying the business adaptability of the extended model, and the specific implementation mode and logical relationship are as follows:

[0142] The API gateway metadata service implements real-time field acquisition of the master data interface through Swagger Hub. The master data management system calls the API of Swagger Hub (such as GET / apis / {api_id} / metadata) to acquire the OpenAPI document of the current master data service interface. The document includes the path of the interface (such as GET / customers), the HTTP method (such as GET), and the response field definition (such as the "schema" attribute in the "responses" section). For example, in the OpenAPI document of the customer master data interface, the system acquires the list of real-time fields returned by the current interface (such as ["customerID", "name"]) by parsing the "properties" field of the document, and uses it as the basis data for interface expansion.

[0143] The interface field expansion script dynamically adjusts the interface definition according to the expansion model metadata. The system parses the newly added attribute (such as "certificateNo", data type "string", constraint "maxLength:20") in the expansion model metadata to generate the interface field expansion script. The script adds the definition of the attribute by modifying the "properties" section of the OpenAPI document:

[0144] After the script is executed, the system calls the update API of Swagger Hub to synchronize the modified OpenAPI document to the gateway, so that the business system can view the latest interface specification in real time through tools such as Swagger UI. For example, after expansion, the response field of the customer interface is added with "certificateNo", and the ERP system can acquire complete customer data when calling the interface.

[0145] The data lineage analysis tool tracks the storage and association of historical data through Apache Atlas. The system calls the lineage query API of Apache Atlas (such as GET / v2 / lineage / {guid}) to acquire the storage path and field association relationship of the historical master data instance. For example, the historical instance of the product master data is stored in the product_v1 table of the Hive data warehouse, and Apache Atlas records the lineage information of the table (such as the data comes from the procurement module of the ERP system, including "productID", "name", "remark" fields). The system extracts the historical fields related to the new attribute (such as the "remark" field, which is used to map to the new "certificateNo" attribute) by parsing the lineage information, and acquires the storage location of the field (such as Hive table "product_v1.remark").

[0146] The data migration script realizes the compatibility of historical data and the extended model through SQL statements. The system generates a data migration script based on the length constraints of the extended model metadata (such as "certificateNo" length ≤ 20). The script uses SELECT statements to extract relevant fields from the historical table and applies the constraint rules.

[0147] The data quality rule library stores business rules through database tables, supports conflict detection and report generation. The rule library table (such as quality_rules) includes "rule_id" (unique rule identifier, such as "R001"), "rule_type" (rule type, such as "UNIQUE" "NOT_NULL"), "rule_expression" (rule expression, such as "customerID UNIQUE") and other fields. The system reads the rules from this table and checks one by one whether the new properties of the extended model conflict with existing rules. For example, if the new property defines "certificateNo UNIQUE", the system queries all "UNIQUE" type rules in the rule library to detect whether there are other fields (such as "productCode") that have defined the same rule. If a conflict is detected (such as "productCode UNIQUE" and "certificateNo UNIQUE" are repeated), the system generates a conflict report, which is pushed to the administrator console. The administrator can adjust the constraint rules of the extended model (such as canceling "certificateNo UNIQUE" definition) or modify the existing rules (such as adjusting the scope of "productCode UNIQUE") according to the report.

[0148] The above steps realize multi-dimensional verification of the extended model in the technical implementation, data compatibility and business compliance levels through the interface extension of the API gateway, historical tracking of data bloodlines, constraint application of migration scripts and conflict detection of rule libraries, providing a reliable basis for the formal implementation of the model.

[0149] In a second aspect, the present application provides a master data management system based on enterprise data governance, which is applied to a master data management method based on enterprise data governance, comprising:

[0150] The demand perception module deploys an event listening component in the business system log, parses the change event and manual demand into a structured extended demand description, and pushes it to the subsequent module through a message queue;

[0151] The dynamic modeling module stores the basic model and extended model metadata, supports version management and metadata query, and provides a basic model metadata interface to the extended configuration module;

[0152] An extension configuration module reads basic model metadata through a visual interface, supports drag-and-drop attribute extension, tree diagram classification extension and relationship diagram association extension, generates extension model metadata and stores the extension model metadata to a dynamic modeling module;

[0153] An adaptation engine module reads the extension model metadata, calls a business system API gateway, a data blood relationship analysis tool and a data quality rule library, performs interface adaptation, data mapping adaptation and rule conflict detection, and outputs an adaptation result; a life cycle management module synchronizes the effective extension model metadata to a service layer, a cleaning layer and a monitoring layer, wherein the service layer is used for main data service interface management, the cleaning layer is used for data cleaning processing, and the monitoring layer is used for data quality monitoring, and is used for extension synchronization and monitoring of the whole life cycle of main data.

[0154] The main data management system based on enterprise data governance provided by the application realizes whole-process management of main data model from demand perception to extension landing through modular design, and each module cooperates, and the specific implementation mode and logical relationship are as follows:

[0155] A demand perception module: capture and integrate business change signals

[0156] The demand perception module is an input entrance of the system, is responsible for capturing dynamic change events of a business system and manually submitted extension requirements, and converts the dynamic change events and the manually submitted extension requirements into structured data for subsequent module processing. The module is realized through the following modes:

[0157] Event listening component deployment: a listening component is deployed in a log source of an enterprise core business system (such as an ERP and a CRM), and specifically includes a MySQL Binlog log (recording a database table structure and data change) of an ERP system and an API calling log (recording a business operation request parameter) of a CRM system. The component matches business change keywords such as “ADD COLUMN” and “update_category” through a regular expression, extracts event context (such as change time, involved system and affected object), and for example, parses “2025-05-20 14:30:00, ERP procurement module, product main data new field” event information from the Binlog log of the ERP.

[0158] Manual requirement collection: a Web form interface (developed based on a vue front-end framework) of a management console is provided, and a business personnel can fill in an extension object (such as “customer main data model”), a new attribute name (such as “environmental protection certification number”) and a classification adjustment range (such as “new energy is added to the customer industry”). After the form is submitted, a Spring Boot backend service performs term compliance verification (calls an enterprise term library interface to verify the attribute name) on the input content, and demand compliance with business standards is realized.

[0159] Structured requirement output: System event context and manual requirements are uniformly converted to JSON format through Kafka message queues (topic: "model_extension_request") including eventType (system / manual), targetModel (target model), and extensionDetails (extension details) fields, and pushed to subsequent modules.

[0160] Dynamic modeling module: Store and manage model metadata

[0161] The dynamic modeling module is the metadata core storage unit of the system, responsible for version management of basic models and extended models, providing reference data for extension configuration. This module is implemented in the following ways:

[0162] Metadata storage: MySQL database is used to store metadata, and the table structure includes fields such as model_name (model name, such as "customer_master"), model_version (version number, such as "1.0"), metadata_json (JSON format metadata descriptor), etc. Database transactions (BEGIN...COMMIT) are used to ensure atomicity when writing data.

[0163] Version management: The model_version field records the iteration status of the model (e.g., from "1.0" to "1.1"), and the parentModel field (e.g., "parentModel": "1.0") associates the extended model with the basic model, supporting historical version queries by model name and version number.

[0164] Interface provision: Provide basic model metadata query interfaces (e.g., GET / model / {model_id} / base) to the extension configuration module, returning the current effective basic model metadata (e.g., attribute list, classification tree, and relationship diagram of customer master data) as the initial basis for extension configuration.

[0165] Extension configuration module: Zero-code implementation of model extension

[0166] The extension configuration module is the interactive core of the system, supporting business personnel to complete model extension with zero coding through a visual interface, generating extended model metadata. This module is implemented in the following ways:

[0167] Visualization interface interaction: The front end is developed based on the vue framework, including three areas of attribute list (showing real-time attributes of the basic model, such as "customerID" "name"), classification tree (showing the classification system of the basic model, such as "customer industry -> existing manufacturing"), and relationship diagram (showing the association relationship of the basic model, such as "customer-supplier"). Business personnel add attributes by drag-and-drop operation (input name, data type and constraint rules), add categories by right-click menu (record version and mapping rules), and add associations by relationship diagram configuration (select objects and define rules). For example, when adding a "certificateNo" attribute to the product model, drag the "add attribute" component and input "certificateNo" "string" "maxLength:20".

[0168] Metadata fragment generation: After the front-end component calls the enterprise terminology library interface to verify the compliance of the attribute name, the attribute extension metadata fragment is generated; the classification extension generates a fragment including version and mapping rules; the association extension generates a fragment including rules.

[0169] Metadata merging and storage: After the extension is completed, call the backend merging interface to merge each fragment with the basic model metadata to generate an "unverified" state of the extended model metadata, and store it in the metadata database of the dynamic modeling module.

[0170] Adaptation engine module: Verify the business adaptability of the extended model

[0171] The adaptation engine module is the verification core of the system, responsible for verifying the feasibility of the extended model in terms of technical implementation, data compatibility and business compliance, and outputting the adaptation result. This module is implemented in the following ways:

[0172] Interface adaptation: Call the Swagger Hub API gateway metadata service to get the real-time field definition of the current master data interface, parse the new attributes of the extended model (such as "certificateNo"), generate interface field extension scripts and update OpenAPI documents, and realize that the business system can get the latest interface specification.

[0173] Rule conflict detection: Read the data quality rules in the database table quality_rules (such as "customerID UNIQUE"), check whether the new attribute conflicts with the existing rules (such as "certificateNo UNIQUE" conflicts with "productCode UNIQUE"), generate a report including the conflict rule ID and reason, and push it to the administrator console for review. If the conflict is resolved, the extended model metadata is marked as "effective".

[0174] Life cycle management module: Synchronize the extended model to the business layer

[0175] The lifecycle management module is the landing execution unit of the system, responsible for synchronizing the effective extension model to the service layer, the cleaning layer, and the monitoring layer, and realizing full lifecycle monitoring. This module is realized by the following ways:

[0176] Service layer synchronization: Publish the model_update topic message through the RabbitMQ message bus, including the extended interface field definition (such as the "certificateNo" added to the returned field of GET / customer), notify the business system (such as ERP, CRM) to subscribe to the topic and trigger local interface adaptation (such as updating the response parsing logic of the API client), and realize that the business system can correctly receive new data.

[0177] Cleaning layer synchronization: Call the Apache Spark data cleaning engine, read the attribute constraint rules of the effective extension model (such as "certificateNo length ≤ 20") from the dynamic modeling module, write the rules into the cleaning task configuration (such as "WHERE LENGTH(certificateNo) ≤ 20"), and automatically apply the rules to the newly accessed or updated master data instances, filter or correct the data that does not meet the constraints.

[0178] Monitoring layer synchronization: Extract the extended attributes (such as "certificateNo") from the dynamic modeling module, add records to the monitoring index configuration table (such as the index "certificateNo missing rate"), collect data and calculate the missing rate through a timing task (such as every day at 0 o'clock), and display it on the Grafana dashboard, supporting business personnel to monitor data quality in real time.

[0179] Final state record: After the synchronization of each layer is completed, update the state of the extension model metadata in the dynamic modeling module to "final effective", record the synchronization timestamp and the involved systems (such as "ERP procurement module, CRM customer management module"), and mark that the extension model is formally put into use. Realize the full process management of master data model from demand capture to business landing, solve the problem of slow model expansion response and multiple manual interventions in existing methods, and support enterprises to quickly adapt to business dynamic changes.

[0180] The present application solves the problem of extending the benchmark by the metadata-driven basic model construction mechanism. First, based on the enterprise general master data object (such as customer, product), the basic model structure including core attributes, classification system, unique identification rule and association is defined, and the JSON format metadata descriptor is stored in the model warehouse supporting version management. The metadata includes model version number, parent model reference and other fields, which provides traceable benchmark for subsequent expansion. For example, the basic model metadata of customer master data clearly defines that "customer ID" is a required string, and "customer type" includes "personal customer" and "enterprise customer" classification. Subsequent expansion can directly add elements based on this benchmark to avoid repeated definition of model framework and shorten the expansion preparation time.

[0181] The present application realizes rapid response to business changes through demand perception and zero coding expansion configuration. On the one hand, event listening components are deployed in the database change log and API call log of ERP, CRM and other business systems, and the event context is extracted by matching keywords such as "new product line" and "adjust customer classification" through regular expressions; on the other hand, a Web form interface is opened to receive manual expansion requirements, and both are converted into structured demand description including event type, target model and expansion details through a message queue. Business personnel drag new attributes, expand categories by right-clicking, and configure associations through relationship diagrams through a visual interface to generate expansion metadata fragments and merge them with the basic model. The whole process does not require code writing, and the time-consuming of manual model adjustment is reduced from hours / days to minutes, significantly improving response efficiency.

[0182] In the master data management scheme of the present application, regular expressions are used as intelligent text recognition tools to automatically capture key change signals in business systems. Its core value is to convert unstructured system logs into machine-readable business events without manual log screening. The specific implementation logic is as follows:

[0183] In the business system log monitoring scenario, the event listening component has multiple pre-defined regular rules built-in, which continuously scan the running logs of ERP, CRM and other systems. For example: when "add product specification" appears in the database change log of ERP, the regular rule "add *(product line|product specification|classification)" will immediately recognize this text pattern and automatically mark it as "product model expansion requirement";

[0184] When the operation log of the CRM system records "customer industry type adjustment" operation, the regular rule "adjust *(customer type|industry classification)" will extract the operation object and change type to generate a structured event record.

[0185] In the demand conversion link, the recognized text information is intelligently disassembled through a regular expression: from the text description "add an environmental protection certification field in the customer master data", the operation object (customer master data), operation type (add field), and field name (environmental protection certification) are automatically disassembled; from the request "add a new energy industry subclass for a supplier", the target object (supplier classification), operation instruction (add subclass), and classification name (new energy) are accurately extracted.

[0186] The application realizes the business landing of the extended model through the adaptation verification and the whole life cycle synchronization. After the extended model is generated, the system automatically extends the main data interface field by calling the API gateway metadata service, detects the rule conflict in combination with the data quality rule library, and is marked as effective after the verification. The effective model is synchronized to the service layer (triggers the interface adaptation of the business system), the cleaning layer (writes the data cleaning rule), and the monitoring layer (adds the quality monitoring index) through the message bus, and the layers perform the extended landing operation in parallel. For example, when the "environmental protection certification number" attribute is added to the product master data, the service layer interface synchronously adds the field, the cleaning layer automatically filters the long data, and the monitoring layer real-time tracks the field missing rate, so that the end-to-end closed-loop management of the extended model from definition to application is realized, and the main data model can be quickly adapted when the enterprise business dynamically changes.

Claims

1. A master data management method based on enterprise data governance, characterized in that, include: Step 1: Define the basic model structure based on the enterprise's general master data object. The basic model structure includes core attributes, classification system, unique identification rules and relationships, and is stored in the form of metadata in a model repository that supports version management. Step 2: Obtain business system change events and manually submitted extended requirements, and convert the events and requirements into structured extended requirements including event type, target master data model and extended detail fields; Step 3: Based on the structured expansion requirements and the basic model metadata stored in the model repository, complete the expansion configuration of the master data model's attributes, classifications, and relationships through a visual interface. Read the basic model metadata defined in the model repository, and then use the real-time attribute list, classification system and relationships of the basic model. Add new attributes in the attribute list area by dragging and dropping, enter the attribute name, data type and constraint rules, link the enterprise terminology library, and generate attribute extended metadata fragments; The real-time classification system of the basic model is displayed in the classification system area using a tree diagram. You can select a node and add a subclass by right-clicking the menu, record the classification version and the mapping rules between the old and new classifications, and generate classification extended metadata fragments. In the relationship area, the existing relationships of the basic model are displayed through the relationship graph component. You can select to add a new associated object, configure the association rules, and generate relationship metadata fragments. After the expansion is completed, the expanded metadata fragments of attributes, categories, and relationships are merged with the basic model metadata to generate expanded model metadata marked as pending verification. Step 4: Perform interface adaptation, data mapping adaptation, and rule conflict detection on the extended model metadata. Once the verification is successful, mark it as effective. Step 5: Synchronize the effective extended model metadata to the service layer, cleaning layer, and monitoring layer. The service layer is used for master data service interface management, the cleaning layer is used for data cleaning and processing, and the monitoring layer is used for data quality monitoring, and for the extension synchronization and monitoring of master data throughout its entire lifecycle.

2. The master data management method based on enterprise data governance according to claim 1, characterized in that, Obtaining business system change events involves deploying event listener components for log collection in the database change logs and API call logs of the business system, and using regular expressions to match business change keywords to extract event context including change time, involved systems, and affected business objects; Obtaining manually submitted extended requirements includes receiving extended objects, new attribute names, and category adjustment ranges filled in by business personnel through an open web form interface of the management console for business personnel to operate; The event context extracted by the event listening component and the manually submitted extension requirements are uniformly converted into the structured extension requirements of event type, target master data model and extension detail fields in step 2 through the message queue.

3. The master data management method based on enterprise data governance according to claim 2, characterized in that, Also includes: Core attributes, classification system, unique identification rules, and related relationships; The basic model structure is stored in the form of a metadata descriptor in JSON format. The metadata descriptor includes the model version number, parent model reference, type field of the mapped basic model structure elements, and constraint field of the corresponding unique identification rules and association relationships. The model repository stores basic model metadata, including historical versions, through database transactions.

4. The master data management method based on enterprise data governance according to claim 3, characterized in that, Also includes: Call the business system API gateway metadata service to obtain the field definitions of the real-time master data interface, parse the newly added attributes in the extended model metadata, automatically generate the interface field extension script, and update the OpenAPI documentation of the master data service interface; Data lineage analysis tools display the topological relationships of master data; Read the data quality rule base, verify whether the newly added attributes of the extended model metadata conflict with existing rules, generate a report including the conflicting rule ID and the reason for the conflict, and push it to the administrator console for review; If no conflict is detected or the conflict has been resolved, the extended model metadata is marked as active.

5. The master data management method based on enterprise data governance according to claim 4, characterized in that, Also includes: The extended master data service interface field definition is published through the message bus, and the business system is notified to subscribe to the topic, triggering the local interface adaptation of the business system. The data cleaning engine reads the attribute constraint rules of the effective extended model metadata from the model warehouse, writes the rules into the cleaning task configuration, and automatically applies the rules to newly connected or updated master data instances. Extract extended attributes of the effective extended model metadata from the model repository, add a new record to the monitoring metric configuration table, collect field data from the master data instance through a scheduled task, calculate the missing rate and display it on the dashboard; after synchronization of each layer is completed, record the final effective status of the extended model.

6. The master data management method based on enterprise data governance according to claim 5, characterized in that, The business systems include ERP and CRM business systems. The event monitoring component is deployed in the MySQL Binlog logs of ERP and the API call logs of CRM. It uses regular expressions to match keywords for business changes such as adding new product lines and adjusting customer classifications, and extracts the event context, including the change time, the systems involved, and the business objects affected. After the event context is parsed by the event listener component, it is pushed to the requirement processing engine through the Kafka message queue. Manually submitted extended requirements are filled in through the web form interface of the management console. The form fields include the extended object, the name of the new attribute, and the scope of category adjustment. After submission, it is converted into a structured extended requirement description in JSON format and pushed to the requirement processing engine through the same Kafka message queue.

7. The master data management method based on enterprise data governance according to claim 6, characterized in that, The core attributes are the basic fields in the basic model structure that describe the master data objects, including customer ID and name; The classification system is the hierarchical division of master data objects in the basic model structure, including customer type and industry hierarchy; The unique identification rules define the generation logic for the unique identifier of master data objects in the basic model structure, including the customer ID generation algorithm; The feature types of the metadata descriptor correspond to the components of the basic model structure, and the constraints include attribute mandatory fields and length limit fields. The model repository uses MySQL database transactions to write basic model metadata, which can be used to query historical versions or real-time versions by model name and version number.

8. The master data management method based on enterprise data governance according to claim 7, characterized in that, The API gateway is used to obtain the field definitions of the real-time master data interface; The interface field extension script modifies the interface return fields and updates the OpenAPI documentation based on the newly added attributes in the extended model metadata. The data migration script extracts data from a table and writes it to a new table using a SELECT statement, applying length constraints to the extended model metadata. The data quality rule base is stored in database tables, including rule types and rule expressions; The conflict report includes the conflict rule ID and the reason for the conflict, which is used by the administrator to review the conflict solution.

9. A master data management system based on enterprise data governance, applied to the master data management method based on enterprise data governance as described in any one of claims 1 to 8, characterized in that, include: The requirement awareness module deploys an event listening component in the business system logs, parses change events and manual requirements into structured extended requirement descriptions, and pushes them to subsequent modules through a message queue; The dynamic modeling module stores metadata for the basic model and extended models, which is used for version management and metadata query, and provides a basic model metadata interface to the extended configuration module. The extended configuration module reads the basic model metadata through a visual interface, which is used for drag-and-drop attribute expansion, tree diagram classification expansion, and relationship graph association expansion. It generates extended model metadata and stores it in the dynamic modeling module. The adaptation engine module reads the extended model metadata, calls the business system API gateway and data quality rule base, performs interface adaptation, data mapping adaptation and rule conflict detection, and outputs the adaptation results. The lifecycle management module synchronizes the effective extended model metadata to the service layer, cleaning layer and monitoring layer. The service layer is used for master data service interface management, the cleaning layer is used for data cleaning processing, and the monitoring layer is used for data quality monitoring. It is used for the extension synchronization and monitoring of master data throughout its entire lifecycle.

Images