A data migration method, system and storage medium based on business object mapping

Abstract

The application discloses a data migration method and system based on business object mapping and a storage medium. Metadata of a source system and a target system is acquired, the metadata covering data table groups, fields and field attributes. Data tables having a correlation relationship in the two systems are integrated into source business objects and target business objects respectively based on business semantics. A mapping relationship between the source business objects and the target business objects is established, data in the source business objects is migrated to the target system based on the mapping relationship and conversion rules, and the migrated data is verified. The efficiency and accuracy of data migration are improved, the consistency of migration structure and business semantics is ensured, and semantic deviation and configuration errors caused by a traditional table-level migration mode are reduced. The application can better adapt to a migration scene of complex business data between heterogeneous systems.

Description

Technical Field

[0001] This invention relates to the field of data migration technology in enterprise information systems, and in particular to a data migration method, system, and storage medium based on business object mapping. Background Technology

[0002] In the process of enterprise digital transformation and information technology upgrade, data migration is a crucial step in achieving system replacement, version iteration, or domestic substitution. When enterprises need to migrate business data from the original system to a new system, they must ensure that the data can completely retain the business logic while maintaining the accuracy and validity of the data to avoid interrupting business processes due to data migration issues. This process directly affects the overall effectiveness of enterprise information technology construction and the stability of business operations. Existing technical solutions mostly use a single data table or independent field as the basic unit of migration operations. They identify the data tables in the source system, then configure the mapping relationship between each field in the data table and the field in the target system, while setting separate rules for data format conversion, value adjustment, etc., and then perform data extraction and loading operations sequentially according to the data tables. However, existing technologies have significant limitations when dealing with complex business data migration. They fail to integrate data tables with relationships in the source system based on business semantics, and only process individual data tables or fields in a scattered manner, which can easily lead to the loss of the original business relationship logic in the migrated data. At the same time, the configuration of mapping relationships and transformation rules needs to be carried out field by field and table by table, which can easily lead to mapping errors due to the omission of relationships between tables. Furthermore, the lack of consistency verification for the migrated data makes it impossible to detect data deviations during the migration process in a timely manner, making it difficult to ensure the accuracy of the migrated data and failing to meet the actual requirements of enterprises for the quality of complex business data migration. Summary of the Invention

[0003] This invention addresses the shortcomings of existing technologies by providing a data migration method, system, and storage medium based on business object mapping. It solves problems such as coarse-grained migration modeling, reliance on manual field mapping, and difficulty in reusing migration logic in existing technologies. This invention uses business objects as migration modeling units, and through metadata parsing, field mapping relationship construction, migration object generation, and migration package organization, achieves structured data migration and template-based reuse between heterogeneous systems. The steps include:

[0004] Step S1: Obtain source system metadata and target system metadata, wherein the metadata includes data table groups, fields and field attributes; based on business semantics, combine at least one data table with a relationship in the source system into a source business object, and combine at least one data table with a relationship in the target system into a target business object.

[0005] Step S2: Establish a mapping relationship between the source business object and the target business object. The mapping relationship includes the correspondence between the source fields contained in the source business object and the target fields contained in the target business object, as well as the conversion rules from the source fields to the target fields.

[0006] Step S3: Among the source business objects to be migrated, determine the execution order based on their primary and foreign key dependencies or business logic dependencies and generate a migration execution sequence. Load the data of the source business objects from the source system to the target system according to the mapping relationship and transformation rules.

[0007] Preferably, the metadata acquisition in step S1 specifically includes the following steps:

[0008] Step S11: Extract structured metadata containing database / table / column identifiers, field types and constraints, and Chinese comments from the source system and the target system respectively; use the primary key and foreign key relationships declared in the metadata as the baseline primary key and foreign key relationships.

[0009] Step S12: If the metadata does not declare or the declaration is incomplete, perform data analysis on the fields of the master data table and determine the candidate primary key containing the composite key through uniqueness detection and functional dependency.

[0010] Step S13: Perform full quantum set verification on the matched pairs of transaction table fields / field groups and candidate primary keys of master data after screening, and use heuristic verification by combining field naming similarity, type compatibility and sampling outer join hit rate; classify the degree of association of candidate relationships according to configurable thresholds and add strongly associated candidate relationships to the baseline primary and foreign key relationships.

[0011] Preferably, establishing the mapping relationship between the source business object and the target business object specifically includes:

[0012] Step S21: Extract the names and Chinese descriptions of the source field and the target field, determine whether the source field is a main data field and has been mapped, if so, directly reference the main data mapping relationship; otherwise, query whether there is a matching historical field mapping record; if so, extract the mapping relationship in the matching historical field mapping record as the recommended mapping relationship for the current source field.

[0013] Step S22: If no matching historical field mapping record is found, then check in the preset general mapping rule library to see if there is a matching general mapping rule. If there is, then use the matching mapping relationship as the recommended mapping relationship for the current source field.

[0014] Step S23: If it does not exist, query in the preset similar word library whether there is a matching similar field. If it exists, obtain the mapping relationship of the matching similar field as the mapping relationship recommended by the current source field.

[0015] Preferably, step S21 specifically includes:

[0016] Search the historical field mapping knowledge base to determine if there are any historical records that have the same source field pointing to the same target field, the same rule type, and whose data type, length, and nullable attributes are compatible with the historical records, or historical records whose field names and Chinese descriptions are consistent with the historical records after being normalized by synonyms and aliases. If they exist, mark the mapping item field in the historical records as a strongly recommended mapping field.

[0017] If not found, normalization is performed on the source and target fields. Based on the source and target field profiles, confirmed mapping records that meet the requirements of table or column names or synonyms, safe conversion of type and length, and consistency of business domain and object tags are retrieved from the historical field mapping knowledge base. Candidate records containing field mapping relationships, rule types, versions, and reuse counts are recalled. After performing structural compatibility checks on the recalled candidate records and removing those that fail the checks, the scores of each candidate record are obtained by inputting into a heuristic scoring model. The highest-scoring item is selected and marked as a strongly recommended mapping field after sorting by the scores.

[0018] Preferably, step S3 includes:

[0019] A data migration package is generated based on the selected multiple source business objects and their corresponding structural layers. The structural layers include the execution sequence of each source business object. The data migration package includes, but is not limited to, the migration execution logic, dependency information, and execution order of the business objects. The data migration package is then sent to the target business object.

[0020] During the migration execution phase, the system organizes and schedules multiple migration objects based on the migration object structure, field mapping configuration, and inter-object dependencies defined in the migration package, forming an executable migration execution sequence. This sequence constrains the order of migration tasks, dependency order, and cross-object execution logic, ensuring consistency and structure in the migration process. The migration package, as a reusable carrier of migration logic, can reuse its object structure, field mapping configuration, and execution order in different systems or projects. New migration package instances can be generated through differentiated configurations to adapt to different migration scenarios.

[0021] This invention also discloses a data migration system based on business object mapping, comprising:

[0022] The data acquisition module is configured to acquire metadata from the source system and metadata from the target system, wherein the metadata includes data table groups, fields, and field attributes; and to combine at least one data table with a relationship in the source system into a source business object based on business semantics, and to combine at least one data table with a relationship in the target system into a target business object.

[0023] The data mapping module establishes a mapping relationship between the source business object and the target business object. The mapping relationship includes the correspondence between the source fields contained in the source business object and the target fields contained in the target business object, as well as the conversion rules from the source fields to the target fields.

[0024] The data migration module determines the execution order and generates a migration execution sequence among the source business objects to be migrated based on their primary and foreign key dependencies or business logic dependencies. It then loads the data of the source business objects from the source system to the target system according to the mapping relationship and transformation rules.

[0025] Preferably, the data acquisition module can also be configured as follows:

[0026] The data processing module is configured to extract structured metadata, including database / table / column identifiers, field types and constraints, and Chinese comments, from the source system and the target system respectively; and to use the primary key and foreign key relationships declared in the metadata as the baseline primary key and foreign key relationships.

[0027] The data detection module is configured to perform data profiling on the fields of the master data table if they are not declared or are not fully declared in the metadata. It then uses statistical analysis based on the distribution of unique field values ​​and functional dependencies to determine candidate primary keys containing composite keys, thereby supplementing the primary key definition and improving the structural identification capability of business objects.

[0028] The data validation module is used to evaluate the relevance of candidate foreign key relationships based on field naming similarity, type compatibility, and outer join hit rate, and to add relationships that meet the threshold to the baseline primary and foreign key relationships.

[0029] The present invention also discloses a data migration apparatus based on business object mapping, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0030] The present invention also discloses a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0031] This invention discloses a data migration method, system, and storage medium based on business object mapping. The method involves acquiring metadata from both the source and target systems, where the metadata includes data table groups, fields, and field attributes. Based on business semantics, at least one data table with a relationship in the source system is combined into a source business object, and at least one data table with a relationship in the target system is combined into a target business object. A mapping relationship is established between the source and target business objects, including the correspondence between source fields in the source business object and target fields in the target business object, and the mapping relationship between the source and target business objects. The conversion rules from the field to the target field; among the source business objects to be migrated, the execution order is determined according to their primary and foreign key dependencies or business logic dependencies, and a migration execution sequence is generated. The data of the source business objects is loaded from the source system to the target system according to the mapping relationship and conversion rules. This invention, through business object modeling, multi-strategy field mapping recommendation, structured modeling of migration objects, and templated reuse of migration packages, makes the migration process have the characteristics of being structured, semantically consistent, having reduced configuration volume, and having high reusability. This improves the efficiency and accuracy of migrating complex business data across systems and enhances the rapid adaptability of the migration solution between different systems or projects.

[0032] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0033] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0034] Figure 1 This is a schematic diagram illustrating the specific process of a data migration method based on business object mapping disclosed in an embodiment of the present invention.

[0035] Figure 2 This is a partial flowchart of step S1 disclosed in an embodiment of the present invention.

[0036] Figure 3 This is a partial flowchart of step S2 disclosed in an embodiment of the present invention.

[0037] Figure 4 This is a partial flowchart of step S3 disclosed in an embodiment of the present invention.

[0038] Figure 5 This is a schematic diagram of the structure of a data migration system based on business object mapping disclosed in an embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0040] Unless otherwise defined, the technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains. The terms “first,” “second,” and similar terms used in the specification and claims of this patent application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms “an” or “a” and similar terms do not indicate a limitation of quantity, but rather indicate the presence of at least one.

[0041] In this embodiment, as shown in the appendix Figure 1 As shown, a data migration method based on business object mapping is disclosed, which may specifically include the following steps.

[0042] Step S1: Obtain source system metadata and target system metadata, wherein the metadata includes data table groups, fields and field attributes; based on business semantics, combine at least one data table with a relationship in the source system into a source business object, and combine at least one data table with a relationship in the target system into a target business object.

[0043] Structured metadata is extracted from the system directories or data dictionaries of the source and target systems. The structured metadata includes data table groups, fields, and field attributes. The data table groups cover multiple related source tables, as well as primary and foreign key relationships used to record the primary and foreign key entity relationships between the source tables. The extracted structured metadata is standardized to unify field identifiers, data type formats, non-null attributes, index information, and column comment content. The identification of primary and foreign key relationships is based on structural association characteristics such as field naming similarity, data type compatibility, and outer join hit rate, which is used to supplement missing structural association relationships in the metadata. Based on business semantics, the data tables with relationships in the source system are filtered and categorized, and at least one related data table is integrated into a source business object. The table group composition corresponding to the source business object and the primary and foreign key relationship logic between each table are clarified. Business semantic classification is performed on the data tables with relationships in the target system, and at least one related data table is integrated into a target business object. The table group structure and primary and foreign key relationship of the target business object are determined simultaneously. In addition, the table group relationship structure between the source business object and the target business object can be adjusted through the drag-and-drop, configuration and preview functions provided by the graphical modeling interface to ensure that each business object is accurately associated with the data tables and primary and foreign key relationships in the corresponding system.

[0044] In another embodiment, migration objects can be modeled by constructing a business object mapping model. This model categorizes, abstracts, and centrally maps the table structures of the source and target systems based on business semantics. A metadata interface is provided, and table structures are imported from both the source and target systems. A set of business objects is generated by categorizing them according to business semantics. This set of business objects serves as the basic unit for migration modeling. Each set of business objects contains a set of related database tables, such as master and child tables, or parent and child table structures, representing a specific business concept, such as sales orders, purchase orders, purchase requisitions, or customer master data. Users can select a target business object on the modeling platform. The system displays the object's standard structure, and the user automatically matches or selects the corresponding table set in the source system. The system guides the user to complete field-level mapping configuration, which includes field name matching, data type parsing, and conversion rules. The entire process involves visual operations and provides an automatic recommendation mechanism, such as recommending mappings based on field name similarity, to help users complete configurations efficiently. Taking the migration of a certain type of business object as an example, the source system business object contains corresponding main tables and sub-tables, and the target system business object contains corresponding main tables. Its modeling requires the user to select the target object on the platform and then the system will display the structure definition. The user can associate the source system business object or the system can automatically match the corresponding main tables and sub-tables. The system recommends preliminary field mappings based on field names and semantics, including scenarios such as single field correspondence and multi-field combination correspondence. Users can choose to add conversion functions, such as date format unification and amount field standardization. The system automatically identifies primary and foreign key logic and displays the table topology in the graphical interface. Finally, the modeling result of the business object is solidified into an executable migration object, which can be scheduled and reused as a component of the migration package.

[0045] During the business object mapping model construction phase, the system categorizes data tables in the source and target systems according to business semantics, forming their respective business object sets. Users can select a target business object and match it with the corresponding business object in the source system based on its structural definition, achieving centralized mapping configuration. Business object modeling encompasses multiple dimensions: table group structure supports grouping multiple related source tables into a single business object; primary and foreign key relationships record the primary and foreign key ER associations between tables, ensuring data integration consistency; field mapping relationships define the correspondence between source and target fields, supporting various mapping forms such as one-to-one, one-to-many, many-to-one, and field combination concatenation; and the target structure mapping path clearly defines the path from source data to specific tables and fields in the target system. The system provides a graphical modeling interface, allowing users to intuitively build and adjust object structures through drag-and-drop, configuration, and preview. The completed migration object includes structural definitions, mapping rules, and transformation logic, and can be used as the basic unit of migration configuration in subsequent migration packages. It can also be imported and reused across projects and systems to improve modeling efficiency and consistency.

[0046] In another embodiment, a multi-stage verification method for the data migration process based on business object mapping is used to verify the data migration process, which may include the following:

[0047] Step S101: Verify the objects to be migrated generated according to the migration requirements. If the first target field does not have a corresponding mapping rule, generate placeholder mapping logic and call the corresponding default value strategy to fill in the information according to the target field label. If the main data on which the sub-table record depends is not ready, write the sub-record and fill in the foreign key after the main data is loaded.

[0048] Specifically, step S101 further includes: validating the objects to be migrated generated according to the migration requirements; if a mismatch between the source and target field types is detected, injecting a type conversion function and calling a preset format cleaning rule to perform format calibration on the converted field types, and then generating replacement source and target field types. If the detection value mapping table does not cover all source values, generating a supplementary list and injecting the set fallback rule, and marking the mapping records that use the fallback to fill source values ​​as low confidence and then filling them into the grayscale monitoring queue.

[0049] Step S102: After the data migration package generated based on the object to be migrated enters the mapping engine for migration data transformation, field-level verification is performed. If there is a value mapping that cannot be matched, the historical mapping cache is searched for matching. If there is still no match, it is marked and a difference list is generated.

[0050] Step S102 includes the following steps.

[0051] Step S1021: Capture each abnormal behavior and enter it into the difference table according to its type. The abnormal behavior includes, but is not limited to, the main object on which the foreign key depends has not been migrated, the field values ​​under the same primary key are different, the foreign key in the sub-table cannot be found in the corresponding primary key in the main table, the required fields required by the target system are missing values, and the set format does not conform to the target system specifications.

[0052] Step S1022: Generate a first compensation task based on the type of abnormal behavior, keep the main migration process running and execute the first compensation task asynchronously.

[0053] If a foreign key in a sub-table cannot be found in the corresponding primary key in the main table, a pre-defined surrogate key is retrieved based on the foreign key in the sub-table and written as a placeholder. Once the corresponding primary key is found, the surrogate key is replaced with the actual primary key. If a required field required by the target system is missing a value, a pre-defined default value is retrieved based on the required field attribute and filled into the corresponding field. If null value handling fails, a pre-defined placeholder value is added, and the corresponding record is delayed until a later batch to complete the process. If the foreign key depends on a main object that has not yet been migrated, resulting in a missing code value, the historical mapping knowledge base or rule base is called to query for a matching code value to fill it. If a matching code value cannot be obtained, a placeholder value is filled, and the corresponding code value is backfilled after the main object migration is detected.

[0054] Step S102 may also include the following:

[0055] Step S1023: After executing the first supplementary task, data verification and abnormal behavior capture are performed again based on the repaired data, and each record with a set problem identifier in the difference table is tracked and reviewed. For records with a secondary backfill identifier, a second compensation task is triggered. The second compensation task includes, but is not limited to, modifying the placeholder value based on the current main object migration completion status.

[0056] Step S103: After the mapping is completed, the return code of the database engine used to return the write result is parsed in real time. If it is a primary key conflict, the deduplication rule library is called to filter and retain the latest or valid record. If it is a foreign key conflict, the foreign key is backfilled after the placeholder is written and marked with a specific identifier and the dependent object is loaded. If the type write fails, the safe pruning function is triggered to process the data and retain the backtracking key.

[0057] In this embodiment, as shown in the appendix Figure 2 As shown, the metadata acquisition in step S1 specifically includes the following steps:

[0058] Step S11: Extract structured metadata containing database / table / column identifiers, field types and constraints, and Chinese comments from the source system and the target system respectively; use the primary key and foreign key relationships declared in the metadata as the baseline primary key and foreign key relationships.

[0059] Structured metadata, including library identifiers, table identifiers, column identifiers, field types, constraints, and Chinese comments, is extracted from the source system metadata and the target system's system directory or data dictionary. The extracted metadata undergoes standardization processing: the naming format of library identifiers, table identifiers, and column identifiers is unified, removing spaces and special characters, and standardizing capitalization according to preset rules; the description of field types is aligned, and for fields of the same type with differing descriptions across different systems, the descriptions are adjusted to be consistent according to general type standards; the semantic content of Chinese comments is standardized, and semantic normalization processing (including text cleaning and extraction of core business phrases) is performed on the Chinese comments for subsequent field semantic analysis; simultaneously, synonym merging and semantic unification of comment content are performed using standard business phrases maintained in a similarity thesaurus. Primary and foreign key relationships explicitly declared through table creation statements or metadata definitions are extracted from the standardized metadata, and these relationships are identified as baseline primary and foreign key relationships and stored in the baseline relationship table of the metadata management database. The source system, declaration method, and extraction time of these baseline primary and foreign key relationships are also recorded.

[0060] Step S12: If the metadata does not declare or the declaration is incomplete, perform data analysis on the fields of the master data table and determine the candidate primary key containing the composite key through uniqueness detection and functional dependency.

[0061] Specifically, if the metadata of the source or target system does not declare primary key and foreign key relationships, or if the declared primary key and foreign key relationships are missing or incomplete, then data profiling operations are performed on the fields in the master data table to generate profile elements such as field name segmentation, data type, cardinality, missing rate, value pattern, and index clues; uniqueness checks and non-null checks are performed on the fields in the master data table in sequence to filter fields that meet the record uniqueness and non-null constraints; the relationships between fields in the master data table are analyzed based on the functional dependency discovery method to identify single fields or combinations of fields that can uniquely identify each record in the table; and the fields or combinations of fields confirmed by uniqueness checks, non-null checks, and functional dependency analysis are determined as candidate primary keys containing composite keys.

[0062] Step S13: Perform full quantum set verification on the matched pairs of transaction table fields / field groups and candidate primary keys of master data after screening, and use heuristic verification by combining field naming similarity, type compatibility and sampling outer join hit rate; classify the degree of association of candidate relationships according to configurable thresholds and add strongly associated candidate relationships to the baseline primary and foreign key relationships.

[0063] Specifically, a preliminary screening is performed on the fields or field groups of the transaction table and the candidate primary keys of the master data to identify potential matching pairs. Full quantum set verification is then performed on the matched pairs to determine whether the value set of the transaction table field or field group is included in the value set of the candidate primary keys of the master data. The naming similarity between the transaction table fields and the candidate primary key fields in the matched pairs is calculated to verify the compatibility of their data types and lengths. A sampling outer join operation is performed on the matched pairs, and the hit rate is calculated. The results of each verification dimension are used as heuristic evidence. Based on a preset configurable threshold, the candidate relationships that have undergone full quantum set verification and heuristic verification are scored and graded according to their degree of association. Candidate relationships that reach a strong association level are selected and added to the baseline primary-foreign key relationships. The association basis and grading results of these candidate relationships are recorded.

[0064] Step S2: Establish a mapping relationship between the source business object and the target business object. The mapping relationship includes the correspondence between the source fields contained in the source business object and the target fields contained in the target business object, as well as the conversion rules from the source fields to the target fields.

[0065] Specifically, the system invokes a field mapping knowledge base that stores confirmed mapping relationships for historical projects. Based on the name, data type, and Chinese comments of the source fields contained in the source business object, it retrieves the target fields contained in the matching target business object. For source and target fields that do not match historical mappings, it invokes a built-in general mapping rule library to match standard field correspondences, while comparing the Chinese descriptions of the two and combining synonyms in a similar thesaurus to determine the field correspondences. For source and target fields with established correspondences, it configures conversion rules, including field preprocessing rules for null value filtering and illegal character removal, value mapping rules for static mapping tables or dynamic dictionaries, and conversion functions for format conversion, field concatenation, and regular expression cleaning. The established field correspondences and conversion rules are displayed through a graphical configuration interface, which supports adjusting field correspondences and modifying conversion rule parameters. After confirming that the configuration content is correct, the system saves the mapping relationship between the source and target business objects and records the configuration version and confirmation information of the mapping relationship.

[0066] By constructing a structured knowledge base mechanism, historical experience, general rules, and semantic relationships are precipitated as knowledge assets. This mechanism includes knowledge base type definitions and usage methods. The similarity thesaurus aims to identify fields with the same business meaning but different expressions. It uses the Chinese names of source and target fields as comparison units, filters high-frequency field phrases and removes generic words, calculates the similarity between fields using a specific method, and groups similar fields into the same similar word group. The system provides a visual interface to display relevant word groups for users to confirm, modify, or remove, and records metadata such as the reviewer, review time, and review status for each word group. The field mapping knowledge base stores confirmed field mapping relationships from historical projects, covering field names, field types, and corresponding rules. It can be reused across systems and projects, via interface queries or knowledge base reuse recommendations. Recommended items require manual confirmation before taking effect. The general mapping rule library contains standard mapping pairs and conversion rules for common fields. This library is configured for automatic alignment across field types, including built-in functions, regular expressions, and standard processing logic such as unit conversions.

[0067] In another embodiment, a multi-strategy fusion recommendation mechanism can be used for field mapping; the multi-strategy fusion recommendation mechanism includes strategies such as master data mapping inheritance, historical mapping reuse, general mapping rule matching, default rule matching mechanism, similar word library matching, and large model matching recommendation.

[0068] The system has a built-in default processing rule library, containing rule sets that require no processing by default and those that are automatically processed. Specific rules include ignored fields, ID generation rules, default value filling rules, and time format conversion. It supports rule expansion and version management, providing standardized support for field processing. During the field mapping and migration rule configuration phase, it automatically or assistedly calls various knowledge bases and corresponding strategies, following a multi-strategy fusion recommendation mechanism to advance the configuration process. Specifically, the multi-strategy fusion recommendation mechanism includes the following:

[0069] The historical mapping reuse strategy includes querying historical mapping configurations based on source field names, data types, and field descriptions. If a complete match or a strongly recommended match is found, the corresponding mapping pair is automatically populated, and the match is marked as a historical recommendation. If no matching results are found in the historical knowledge base, the process transitions to the rule recommendation stage, calling the general mapping rule library to complete the recommendation work for standard fields. If the current field matches the default processing rule, the corresponding default logic is automatically applied.

[0070] The similarity word matching strategy includes extracting the Chinese descriptions of the source and target fields, comparing these descriptions with word groups in the similarity word library, and determining the matching result based on preset strong and weak rules. The strong rules include a complete match if the Chinese descriptions of two fields completely match the same word; the weak rules include a weak match if the source and target fields each match different words within the same similar word group, and both match words with the group's central word, meeting preset similarity standards. When these matching conditions are met, the system automatically generates a recommendation mapping pair and marks it as a similar word recommendation. Users can view the recommendation criteria and matching logic in the configuration interface and complete manual confirmation.

[0071] The large model matching recommendation strategy includes calling the general large model API interface to achieve field matching, combining prompt words to guide the large model to deeply analyze the Chinese name, English name, data type and business semantics of the field, output the optimal matching suggestion, and mark the field status. At the same time, the matching item is marked as a large model recommendation.

[0072] In another embodiment, the source field, target field, and Chinese description are extracted. It is determined whether they are master data fields and have been mapped. If so, the master data mapping is referenced. Otherwise, it is determined whether there are historical field mapping records. If so, the historical mapping is recommended. Otherwise, it is determined whether they match the general mapping rule library. If so, the general mapping is recommended. Otherwise, it is determined whether they match the default rule library. If so, the default rule is recommended. Otherwise, it is determined whether they match the similar word library. If so, candidate fields are recommended. If none of these conditions are met, the large model matching is used to assist in the output of field mapping suggestions containing the recommendation reason and matching path.

[0073] The multi-strategy fusion recommendation mechanism, through the synergistic effect of multi-level knowledge base calling process and large model matching strategy, integrates semantic intelligence and rule automation capabilities to achieve semi-automation of field-level mapping configuration, which can significantly improve configuration efficiency, reduce manual operation costs, and continuously accumulate reusable knowledge assets.

[0074] In this embodiment, the master data mapping inheritance strategy includes, when a field belongs to the master data field in terms of business semantics, the system identifies its association with the master data business object, automatically reuses the field mapping rules and conversion logic already defined for that master data object, and ensures the consistency of master data-related field mappings. The historical mapping reuse includes: the system builds and maintains a mapping knowledge base, storing field mapping relationships defined by users in different projects or modules, and quickly matching based on features such as field name, table name, and data type. If a matching historical configuration exists, the mapping relationship is directly referenced, reducing redundant configuration work. The default rule matching mechanism includes: a built-in preset rule set containing default processing logic that requires no manual intervention, such as automatically ignoring specific identifier and timestamp fields, and automatically generating identifier fields that need to be generated using a preset algorithm; the similarity word library matching includes: the system maintains synonym phrases of high-frequency business terms, calculates the similarity between the Chinese descriptions of the source field and the target field using a similarity algorithm, and if both belong to the same synonym phrase and the similarity reaches a set threshold, they are listed as mapping candidates; the large model matching recommendation includes: based on the existing mapping knowledge base and historical data, calling the general large model interface, and combining the prompt word engineering to guide the model to compare and analyze the semantic connotation, contextual association, and structural features of the newly added fields, generating mapping recommendation results containing priority scores, marking them as large model recommendations, and supporting manual confirmation and modification.

[0075] In this embodiment, the master data mapping inheritance may specifically include the following steps.

[0076] Step S201: Collect metadata information of the field to be configured, call the preset master data business semantic feature library, compare the collected field metadata information with the master data semantic feature library, and determine whether the field description contains feature words of master data type; at the same time, retrieve the business classification tag of the data table to which the field belongs to confirm whether the data table belongs to the master data business domain; combine the semantic matching result and the data table business classification tag to generate a judgment conclusion on whether the field to be configured belongs to the master data field, and mark the judgment basis at the same time.

[0077] Step S201 specifically includes the following steps.

[0078] Step S2011: Access the system metadata dictionary or data table structure management module to extract the metadata information of the field to be configured. The metadata information includes the Chinese description text of the field, the complete identifier of the data table to which it belongs, the business description implied in the field comments, and the identifier of the associated module of the field in the business process. At the same time, read the data type, non-null constraint, whether it carries foreign key mark and other attributes of the field, integrate them into a structured metadata set and store it in the temporary configuration cache area.

[0079] Step S2012: Invoke the pre-configured master data business semantic feature library. The feature library stores typical semantic identifiers according to master data type and stores the master data category labels associated with each semantic identifier. Based on the business module identifier associated with the field to be configured, load the semantic feature subset corresponding to the master data type that the business module may involve.

[0080] Step S2013: The collected Chinese descriptions and annotations of the fields are segmented to remove generic words and extract core words; the core words are then compared bidirectionally with the semantic feature subset of the loaded master data. The bidirectional comparison includes: performing exact matching to determine if there are completely identical semantic identifiers; if no match is found, performing fuzzy matching based on a semantic similarity algorithm to determine if the similarity between the core words and the feature words reaches a preset threshold; and simultaneously recording the matching results, including whether a match was found, the main data type of the matched words, the matched feature words, and the similarity value, to form a semantic matching report.

[0081] Step S2014: Query the metadata attributes of the data table to which the field to be configured belongs, extract the preset business category tags, and simultaneously access the registry of the master data management module to check whether the data table has been registered as a master data management object and whether it is associated with a master data synchronization task; combine the business category tags of the data table with the master data registration information to determine whether the data table belongs to the master data business domain, and generate a data table affiliation determination result.

[0082] Step S2015 involves a comprehensive analysis of the semantic matching report and the data table attribution determination results: If the semantic matching matches the main data feature words and the similarity meets the standard, and the data table belongs to the main data business domain, then the field to be configured is directly determined to be a main data field; if only the semantic matching meets the standard but the data table is not in the main data domain, or only the data table is in the main data domain but the semantics do not match, the software further searches for the associated configuration records of the field and the main data object to supplement the determination basis; a determination conclusion is generated and key basis is marked in the conclusion, including the matched main data semantic words, the business classification tags of the data table, and the supplemented searched associated configuration records, forming a traceable determination log.

[0083] Step S202: Retrieve the association between the field and the master data business object, and confirm the master data business object to which the foreign key association points.

[0084] Specifically, the associated attribute information of the field to be configured is extracted, including whether the field carries a foreign key constraint marker, the reference path of the associated data table, and historical association configuration records; the system metadata dictionary is queried to retrieve data table entries that have foreign key constraint associations with the field; the associated data table entries are compared with the list of master data business objects to filter associated data tables belonging to the master data category; the dependency graph is invoked to trace the indirect association links between the field and the master data business objects, excluding association nodes of non-master data types; the validity of the association links is verified, and it is checked whether the association relationship conforms to the master data reference rules of the current business scenario; the unique and valid master data business object pointed to by the foreign key association is determined, and the identification information and association level of the master data business object are recorded.

[0085] Step S203: Extract the field mapping relationships and corresponding conversion logic that have been configured in the master data business object; apply the extracted master data field mapping rules and conversion logic to the mapping configuration of the field to be configured.

[0086] Specifically, the system locates the configuration storage path of the master data business object and accesses the mapping configuration library corresponding to the master data business object; it reads the field mapping records marked as effective in the configuration library, including the source system field name, target system field name, and field priority; it extracts the conversion logic configuration associated with each mapping record, covering data type conversion rules, value mapping lookup tables, format standardization rules, and business validation logic; it retrieves the auxiliary configurations that the conversion logic depends on, such as unit conversion coefficients, encoding mapping dictionaries, and default value filling strategies; it structurally integrates the extracted mapping relationships and conversion logic, organizes them into a configuration dataset according to the corresponding field dimension, and labels the effective version and applicable scenario of each configuration; and it checks the integrity of the configuration data to ensure that there are no missing conversion logic steps or associated configuration items.

[0087] Establish an association mapping between the currently configured field and the master data mapping rules, matching the corresponding master data mapping entries based on the field identifier and semantic attributes; synchronize the correspondence between the source and target fields of the master data field to the mapping configuration interface of the current field, automatically filling in the matching items of the source and target fields; import the transformation logic in the master data mapping rules, embedding it into the mapping processing flow of the current field in the execution order, including pre-data validation logic, intermediate transformation processing steps, and post-result normalization rules; associate the auxiliary configuration resources that the transformation logic depends on, such as loading the master data value mapping dictionary and format conversion template into the configuration environment of the current field; perform compatibility verification on the applied mapping configuration, checking whether the transformation logic is compatible with the data type, length limit, and business constraints of the current field; lock the applied mapping rules and transformation logic parameters, generate the preliminary mapping configuration result of the currently configured field, and mark the configuration source as master data inheritance.

[0088] In this embodiment, as shown in the appendix Figure 3 As shown, establishing the mapping relationship between the source business object and the target business object specifically includes:

[0089] Step S21: Extract the names and Chinese descriptions of the source field and the target field, determine whether the source field is a main data field and has been mapped, if so, directly reference the main data mapping relationship; otherwise, query whether there is a matching historical field mapping record; if so, extract the mapping relationship in the matching historical field mapping record as the recommended mapping relationship for the current source field.

[0090] Specifically, the system extracts the names and Chinese descriptions of source and target fields from the metadata corresponding to the source and target business objects. The extracted field names undergo standardization processing, including case unification and whitespace removal. The Chinese descriptions are segmented to extract key semantics. The system then calls the master data object association table and business domain tag library, combining the business affiliation of the source field's data table and the semantic information in the field comments to determine if the source field belongs to the master data category and has completed mapping configuration. If the source field is determined to be a master data field and has completed mapping, the system directly calls the mapping configuration file corresponding to the master data object, referencing the correspondence between the source and target fields and the preset conversion rules recorded therein. If the source field is determined to be a non-master data field or has not completed mapping, the system accesses the field mapping knowledge base storing historical configurations, performing a matching query based on the source field's name, data type, and Chinese description segmentation results against similar information in the target field. If a historical field mapping record that completely matches or meets the correlation requirements is found, the system extracts the correspondence between the source and target fields and the conversion rule parameters from that record, using it as the recommended mapping relationship for the current source field, and marking the historical project source and past reuse records of this recommended mapping relationship.

[0091] In this embodiment, step S21 specifically includes: searching the historical field mapping knowledge base, determining whether there is a historical record that has the same source field pointing to the same target field, the same rule type, and the data type, length, nullable attribute of the current field is compatible with the historical record, or a historical record whose field name and Chinese description are consistent with the historical record after being normalized by synonyms and aliases. If there is such a historical record, the mapping item field in the historical record is marked as a strongly recommended mapping field.

[0092] This process involves accessing the historical field mapping knowledge base and retrieving all historical mapping records stored therein. Key information about the current source and target fields is extracted, including the source system identifier, target system identifier, mapping rule type, and the data type, length, and nullable attribute of the current field. The process checks if any historical mapping record meets any of the following conditions: the source field points to the same target field, the mapping rule type is the same, and the data type, length, and nullable attribute of the current field are compatible with the corresponding field attributes in the historical record. After performing synonym replacement and alias normalization on the names and Chinese descriptions of the current source and target fields, the processing result is completely consistent with the field name and Chinese description in a certain historical record. If a historical record meets any of the above conditions, the mapping item field in that historical record is extracted, marked as a strongly recommended mapping field, and the historical record number and attribute compatibility verification result corresponding to the strongly recommended mapping field are recorded.

[0093] If not found, normalization is performed on the source and target fields. Based on the source and target field profiles, confirmed mapping records that meet the requirements of table or column names or synonyms, safe conversion of type and length, and consistency of business domain and object tags are retrieved from the historical field mapping knowledge base. Candidate records containing field mapping relationships, rule types, versions, and reuse counts are recalled. After performing structural compatibility checks on the recalled candidate records and removing those that fail the checks, the scores of each candidate record are obtained by inputting into a heuristic scoring model. The highest-scoring item is selected and marked as a strongly recommended mapping field after sorting by the scores.

[0094] If no historical records meeting the above conditions exist, normalization processing is performed on the source and target fields, including standardizing the case of field names, removing whitespace characters, standardizing synonyms and aliases, and aligning time and numerical formats. Source and target field profiles are extracted, where the source field profile includes field name segmentation, data type, business domain label, and object identifier, and the target field profile includes similar information. Based on these profiles, confirmed mapping records are retrieved from the historical field mapping knowledge base, with search conditions set as identical or synonymous table or column names, safe conversion between data types and lengths, and consistent business domain and object labels. Candidate records are recalled from the search results; each candidate record includes field mapping relationships, rule types, and... Version information, past reuse count, confirmer identity, and confirmation time are recorded. Structural compatibility checks are performed on the recalled candidate records, including data type logical conversion feasibility, whether the target field length meets the source field data storage requirements, nullable attribute matching degree, and the validity of dependent code tables and unit configurations. Candidate records that fail the checks are removed. The candidate records that pass the checks are input into a heuristic scoring model. The model calculates the score for each candidate record based on field name and description similarity, structural compatibility, past reuse frequency, and business domain consistency. Candidate records are sorted from highest to lowest score. The candidate record with the highest score is selected, and its corresponding mapping field is marked as a strongly recommended mapping field. The scoring details and sorting results for this candidate record are recorded.

[0095] Step S22: If no matching historical field mapping record is found, then check in the preset general mapping rule library to see if there is a matching general mapping rule. If there is, then use the matching mapping relationship as the recommended mapping relationship for the current source field.

[0096] If no matching historical field mapping record is found after step S21, a preset general mapping rule library is invoked. This library contains standard mapping pairs for common fields, cross-field type secure conversion logic, and general business processing rules. The names, business semantic descriptions, data types, and business domain labels of the source and target fields are extracted. Based on this information, matching general mapping rules are retrieved from the general mapping rule library. The retrieval dimensions include the semantic relevance of field names, data type matching, and consistency of business domain affiliation. If a matching general mapping rule is found, that is, the source field features corresponding to the rule are consistent with the current source field features, the target field features match the current target field features, and the conversion logic contained in the rule is applicable to the current scenario, the source field and target field mapping relationship corresponding to the general mapping rule is extracted and used as the recommended mapping relationship for the current source field. The rule number and applicable scenario description of the general mapping rule are recorded.

[0097] Step S23: If it does not exist, query in the preset similar word library whether there is a matching similar field. If it exists, obtain the mapping relationship of the matching similar field as the mapping relationship recommended by the current source field.

[0098] If no matching general mapping rule is found in the preset general mapping rule library after step S22, a preset similar word library is called. This word library stores similar word groups and field relationships within the word groups that have been manually reviewed and confirmed. The Chinese descriptions of the source field and the target field are extracted, and word segmentation is performed on the Chinese descriptions to remove meaningless words and standardize the expression. Based on the processed Chinese descriptions, the similar word library is searched to determine whether there are matching similar fields that belong to the same similar word group as the Chinese description of the source field and also belong to the same similar word group as the Chinese description of the target field. If the above-mentioned matching similar fields are found, the mapping relationship corresponding to the matching similar field is extracted and used as the recommended mapping relationship for the current source field. The similar word group identifier, Chinese description matching basis, and word library review information of the matching similar field are recorded.

[0099] Step S3: Among the source business objects to be migrated, determine the execution order based on their primary and foreign key dependencies or business logic dependencies and generate a migration execution sequence. Load the data of the source business objects from the source system to the target system according to the mapping relationship and transformation rules.

[0100] Specifically, based on the established mapping relationship between source and target business objects and preset conversion rules, the built-in execution engine is invoked to parse the migration object structure, field mapping configuration, data filtering conditions, and various conversion logics, generating an executable data extraction and loading job. The execution engine extracts structured data from the data table associated with the source business object and performs conversion operations such as field preprocessing, format standardization, value mapping conversion, function operation, and data combination on the extracted source data according to the conversion rules, so that the converted data meets the structural requirements and data type constraints of the target business object. The system also supports selecting different execution modes according to the migration strategy, including full migration, incremental migration, batch migration, and field-level or record-level migration based on condition filtering, to adapt to the migration needs of different business scenarios. The transformed data is loaded into the data table corresponding to the target business object, which is located in the target system. The migration trajectory information of each piece of data is recorded synchronously. During the data migration process, static verification can be performed first to check the integrity of the mapping configuration, the matching degree of field attributes, and the validity of dependencies. Dynamic monitoring is started during migration execution to capture data transformation anomalies and write errors, and record the batch execution status and failure reasons. After the migration is completed, a consistency comparison is performed on the data of the source business object and the target business object, including primary key row count verification, foreign key referential integrity verification, key field value sampling comparison, and business key indicator verification, to confirm that the data in the target business object is consistent with the data in the source business object, and a verification report and difference record are generated.

[0101] In this embodiment, step S3 may further include: generating a data migration package based on the selected multiple source business objects and their corresponding structural layers, wherein the structural layers include the execution sequence of each source business object; the data migration package includes, but is not limited to, the migration execution logic, dependency information, preset verification items, and execution order of the business objects; and sending the data migration package to the target business object.

[0102] By calling the migration package configuration interface, multiple migration objects to be migrated are selected. Combining business domain classification and the previously constructed dependency graph, the structural hierarchy of each source business object is sorted out. Based on the dependency graph analysis, the primary and foreign key relationships and execution sequence logic between each source business object are determined to improve the structural hierarchy. The migration process corresponding to each source business object is parsed, and the migration execution logic including data extraction logic, transformation rule calling logic, and target path loading is encapsulated. The pre-dependency and post-association information between each source business object is integrated as dependency information. The execution order of each source business object is determined according to the execution sequence, and a data migration package is generated. Through the system's built-in cross-system data transmission interface, the generated data migration package is sent to the receiving module corresponding to the target business object, and the version information, adaptation scope description, and other meta-information of the migration package are transmitted synchronously, so that the target end can load the migration package and build the corresponding migration execution process.

[0103] In this embodiment, step S3 may further include: template-encapsulating the generated migration package to support cross-industry, cross-enterprise, and cross-project reuse of migration configurations. The system provides a migration package template library for storing standardized migration package templates. Each migration package template consists of one or more migration objects and includes object structure definitions, field mapping configurations, conversion rules, and execution order and dependencies between objects. Migration package templates support importing and exporting between different migration projects, enabling cross-system sharing of migration configuration assets.

[0104] The system establishes a multi-dimensional tagging system for classifying and managing migration package templates based on different dimensions such as industry, enterprise, and business scenario. This tagging system includes general tags, industry tags, enterprise tags, and business scenario tags. Users can search and filter templates based on tags to quickly locate templates suitable for the current migration scenario. Users can inherit from existing templates, adjust field mappings, modify conversion rules, or add or delete migration objects to generate migration package instances suitable for specific industry or enterprise migration needs.

[0105] Within the migration package, the system supports a reference inheritance mechanism for migration objects, enabling them to reuse field mapping relationships, value mapping configurations, and conversion rules from other objects. Inheritance methods include direct copying, suggestive inheritance, and automatic recommended inheritance based on similarity comparison. For cross-system scenarios, the system abstracts source data objects into standard objects and establishes cross-system inheritance relationships to support the reuse of migration objects between different subsystems. While establishing inheritance relationships, the system generates an object inheritance graph to display the inheritance chain between objects and support subsequent change impact analysis.

[0106] In this embodiment, as shown in the appendix Figure 4 As shown, step S3 further includes:

[0107] Step S31: Verify the objects to be migrated generated according to the migration requirements. If the first target field does not have a corresponding mapping rule, generate placeholder mapping logic and call the corresponding default value strategy to fill in the information according to the target field label. If the main data on which the sub-table record depends is not ready, write the sub-record and mark it, and fill in the foreign key after the main data is loaded.

[0108] The pre-defined verification module is invoked to perform verification operations on the objects to be migrated. The verification dimensions cover the completeness of the field mapping rules of the objects to be migrated and the ready status of master data dependencies. If the first target field is found to be without a corresponding mapping rule during the verification process, the auto-completion logic is triggered to generate placeholder mapping logic that includes null value handling and default value filling. At the same time, the field tags associated with the first target field are read, and the preset default value strategy is invoked according to the tag type to perform the completion operation on the missing configuration information of the target field. If it is detected that the master data on which the sub-table record depends has not been fully loaded, the sub-table record is written to the intermediate storage area, and the foreign key status of such sub-table records is marked as pending. The master data loading status monitoring process is started. After the master data loading completion signal is detected, the master data primary key information is retrieved, and the foreign key information backfilling operation is performed on the marked sub-table records.

[0109] Step S32: After the data migration package generated based on the object to be migrated enters the mapping engine for migration data conversion, field-level verification is performed. If there is a value mapping that cannot be matched, the historical mapping cache is searched for matching. If there is still no match, it is marked and a difference list is generated. If there is a null value that is not handled, a preset placeholder value is added and the corresponding record is delayed to be completed in a subsequent batch.

[0110] Specifically, after the data migration package generated based on the object to be migrated enters the mapping engine, the mapping engine completes the migration data transformation according to the preset transformation rules, and then calls the field-level validation module to perform validation on the transformation results. If the validation finds that there are cases where value mappings cannot be matched, the historical mapping cache query mechanism is triggered to call the past value mapping records stored in the historical mapping cache for matching query. If no matching mapping relationship is found after the query, the value mapping records that cannot be matched are specially marked, and a difference list containing information on unmatched values, the field identifier, and the transformation time is generated. If the validation finds that null value handling is not hit, that is, the preset null value handling rules do not cover the current null value scenario, the preset placeholder value is automatically added to the corresponding field, and the record containing the null value field is marked as a delayed processing state and allocated to the subsequent migration batch, and the information completion operation is completed when the subsequent batch is executed.

[0111] Step S33: After the mapping is completed, the return code used by the database engine to return the write result is parsed in real time. If it is a primary key conflict, the deduplication rule library is called to filter and retain the latest or valid record. If it is a foreign key conflict, the foreign key is backfilled after the placeholder is written and marked with a specific identifier and the dependent object is loaded. If the type write fails, the safe pruning function is triggered to process the data and retain the backtracking key.

[0112] After data mapping is complete, the system receives the write results from the database engine in real time, parses the return codes in the results, and identifies the status type of the write operation. If the parsing determines that the return code corresponds to a primary key conflict scenario, the system immediately calls the deduplication rule library, retrieves the preset primary key conflict handling strategy in the library, and filters and retains the latest generated record or the record that meets the business validity requirements according to the strategy. At the same time, the conflicting records that are removed are stored in the conflict log library. If the parsing determines that the return code corresponds to a foreign key conflict scenario, the system first writes the current record to the target data table according to the placeholder logic, marks the record with a specific identifier for identifying foreign key conflicts, and starts the dependency object loading status monitoring task. After the dependency object is detected to be loaded, the foreign key information of the dependency object is retrieved, and the foreign key backfill operation is performed on the record marked with the specific identifier. If the parsing determines that the return code corresponds to a write failure scenario, the system triggers the preset security pruning function, performs compliance processing on the data that does not meet the target field type requirements through the function, retains the backtracking key for tracing the original data during the processing, re-attempts to write the processed data, and records the correspondence between the original data and the processed data in the data processing log.

[0113] In this embodiment, as shown in the appendix Figure 5 As shown, a data migration system based on business object mapping is also disclosed, including:

[0114] Data acquisition module 1 is configured to acquire metadata of the source system and metadata of the target system, wherein the metadata includes data table groups, fields and field attributes; and to combine at least one data table with a relationship in the source system into a source business object based on business semantics, and to combine at least one data table with a relationship in the target system into a target business object.

[0115] Data mapping module 2 is configured to establish a mapping relationship between the source business object and the target business object. The mapping relationship includes the correspondence between the source fields contained in the source business object and the target fields contained in the target business object, as well as the conversion rules from the source fields to the target fields.

[0116] Data migration module 3 determines the execution order and generates a migration execution sequence among the source business objects to be migrated based on their primary and foreign key dependencies or business logic dependencies, and loads the data of the source business objects from the source system to the target system according to the mapping relationship and transformation rules.

[0117] In this embodiment, the data acquisition module 1 can also be configured as:

[0118] The data processing module is configured to extract structured metadata, including database / table / column identifiers, field types and constraints, and Chinese comments, from the source system and the target system respectively; and to use the primary key and foreign key relationships declared in the metadata as the baseline primary key and foreign key relationships.

[0119] The data detection module is configured to perform data analysis on the fields of the main data table and determine candidate primary keys containing composite keys by uniqueness detection and functional dependency if the metadata does not declare or the declaration is incomplete.

[0120] The data verification module is configured to perform full set verification on the matched pairs of transaction table fields / field groups and candidate primary keys of the master data after screening, and to provide heuristic evidence by combining field naming similarity, type compatibility and sampling outer join hit rate; to classify the degree of association of candidate relationships according to configurable thresholds and to add strongly associated candidate relationships to the baseline primary and foreign key relationships.

[0121] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. Regarding the data migration system based on business object mapping disclosed in the embodiments, since it corresponds to the data migration method based on business object mapping disclosed in the embodiments, its description can be simplified, and its specific implementation can be referred to the foregoing method embodiments.

[0122] In other embodiments, a data migration apparatus based on business object mapping is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the various steps of the data migration method based on business object mapping as described in the above embodiments.

[0123] In another embodiment, if the above-described data migration method based on business object mapping is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above-described embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various embodiments of the data migration method based on business object mapping. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory, a random access memory, an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0125] In summary, the above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should be covered by the present invention.

Claims

1. A data migration method based on business object mapping, characterized in that, Includes the following steps: S1. Obtain source system metadata and target system metadata, wherein the metadata includes data table groups, fields and field attributes; based on business semantics, combine at least one data table with a relationship in the source system into a source business object, and combine at least one data table with a relationship in the target system into a target business object; S2, establish a mapping relationship between the source business object and the target business object. This mapping relationship includes the correspondence between source fields contained in the source business object and target fields contained in the target business object, as well as the conversion rules from the source fields to the target fields; specifically, it includes: S21, extract the names and Chinese descriptions of the source field and the target field, determine whether the source field is a main data field and has been mapped, if so, directly reference the main data mapping relationship, otherwise query whether there is a matching historical field mapping record; if so, extract the mapping relationship in the matching historical field mapping record as the recommended mapping relationship for the current source field; S22. If no matching historical field mapping record is found, then check in the preset general mapping rule library to see if there is a matching general mapping rule. If there is, then use the matching mapping relationship as the recommended mapping relationship for the current source field. S23, if it does not exist, then query in the preset similar word library whether there is a matching similar field. If it exists, then obtain the mapping relationship of the matching similar field as the mapping relationship recommended by the current source field. S3 determines the execution order among the source business objects to be migrated based on their primary and foreign key dependencies or business logic dependencies and generates a migration execution sequence, loading the data of the source business objects from the source system to the target system according to the mapping relationship and transformation rules.

2. The data migration method based on business object mapping according to claim 1, characterized in that, Obtaining source system metadata and target system metadata involves the following steps: S11: Extract structured metadata containing database / table / column identifiers, field types and constraints, and Chinese comments from the source system and the target system respectively; use the primary key and foreign key relationships declared in the metadata as the baseline primary key and foreign key relationships; S12, If the metadata does not declare or the declaration is incomplete, perform data analysis on the fields of the master data table and determine the candidate primary key containing the composite key through uniqueness detection and functional dependency; S13 performs full quantum set verification on the matched pairs of transaction table fields / field groups and candidate primary keys of master data after screening, and provides heuristic evidence by combining field naming similarity, type compatibility and sampling outer join hit rate; classifies the degree of association of candidate relationships according to configurable thresholds and adds strongly associated candidate relationships to the baseline primary and foreign key relationships.

3. The data migration method based on business object mapping according to claim 2, characterized in that, Step S21 specifically includes: Search the historical field mapping knowledge base to determine whether there are historical records that have the same source field pointing to the same target field, the same rule type, and whose data type, length, nullable attribute of the current field are compatible with the historical records, or historical records whose field name and Chinese description are consistent with the historical records after being normalized by synonyms and aliases. If they exist, mark the mapping item field in the historical records as a strongly recommended mapping field. If not found, normalization is performed on the source and target fields. Based on the source and target field profiles, confirmed mapping records that meet the requirements of table or column names or synonyms, safe conversion of type and length, and consistency of business domain and object tags are retrieved from the historical field mapping knowledge base. Candidate records containing field mapping relationships, rule types, versions, and reuse counts are recalled. After performing structural compatibility checks on the recalled candidate records and removing those that fail the checks, the scores of each candidate record are obtained by inputting into a heuristic scoring model. The highest-scoring item is selected and marked as a strongly recommended mapping field after sorting by the scores.

4. The data migration method based on business object mapping according to claim 3, characterized in that, Step S3 includes: A data migration package is generated based on the selected multiple source business objects and their corresponding structural layers. The structural layers include the execution sequence of each source business object. The data migration package includes, but is not limited to, the migration execution logic, dependency information, pre-defined verification items, and execution order of the business objects. The data migration package is then sent to the target business object.

5. The data migration method based on business object mapping according to claim 4, characterized in that, Step S3 further includes: S31, validate the objects to be migrated generated according to the migration requirements. If the first target field does not have a corresponding mapping rule, generate placeholder mapping logic and call the corresponding default value strategy to fill in the information according to the target field label. If the main data that the sub-table record depends on is not ready, write the sub-record and mark it, and fill in the foreign key after the main data is loaded. S32, after the data migration package generated based on the object to be migrated enters the mapping engine for migration data conversion, field-level verification is performed. If there is a value mapping that cannot be matched, the historical mapping cache is searched for matching. If there is still no match, it is marked and a difference list is generated. If there is a null value that is not handled, a preset placeholder value is added and the corresponding record is delayed to be completed in a subsequent batch. S33, after the mapping is completed, the return code used by the database engine to return the write result is parsed in real time. If it is a primary key conflict, the deduplication rule library is called to filter and retain the latest or valid record. If it is a foreign key conflict, the foreign key is backfilled after the placeholder is written and marked with a specific identifier and the dependent object is loaded. If the type write fails, the safe pruning function is triggered to process the data and retain the backtrack key.

6. A data migration system based on business object mapping, characterized in that, include: The data acquisition module is configured to acquire source system metadata and target system metadata, wherein the metadata includes data table groups, fields and field attributes; Based on business semantics, at least one data table with a relationship in the source system is combined into a source business object, and at least one data table with a relationship in the target system is combined into a target business object. The data mapping module is configured to establish a mapping relationship between the source business object and the target business object. This mapping relationship includes the correspondence between source fields contained in the source business object and target fields contained in the target business object, as well as the conversion rules from the source fields to the target fields. The module extracts the names and Chinese descriptions of the source and target fields, determines whether the source field is a primary data field and has already been mapped, and if so, directly references the primary data mapping relationship; otherwise, it queries whether a matching historical field mapping record exists. If such a record exists, it extracts the mapping relationship from the matching historical field mapping record as the recommended mapping relationship for the current source field. If no matching historical field mapping record is found, the system will search the preset general mapping rule library for a matching general mapping rule. If a matching mapping rule exists, the matching mapping relationship will be used as the recommended mapping relationship for the current source field. If it does not exist, then query in the preset similar word library whether there is a matching similar field. If it exists, then obtain the mapping relationship of the matching similar field as the mapping relationship recommended by the current source field. The data migration module determines the execution order and generates a migration execution sequence among the source business objects to be migrated based on their primary and foreign key dependencies or business logic dependencies. It then loads the data of the source business objects from the source system to the target system according to the mapping relationship and transformation rules.

7. The data migration system based on business object mapping according to claim 6, characterized in that, The data acquisition module can also be configured to: The data processing module is configured to extract structured metadata, including database / table / column identifiers, field types and constraints, and Chinese comments, from the source system metadata and the target system metadata, respectively; and to use the primary key and foreign key relationships declared in the metadata as the baseline primary key and foreign key relationships. The data detection module is configured to perform data analysis on the fields of the main data table and determine candidate primary keys containing composite keys by uniqueness detection and functional dependency if the data is not declared or is incomplete in the metadata. The data verification module is configured to perform full set verification on the matched pairs of transaction table fields / field groups and candidate primary keys of the master data after screening, and to provide heuristic evidence by combining field naming similarity, type compatibility and sampling outer join hit rate; to classify the degree of association of candidate relationships according to configurable thresholds and to add strongly associated candidate relationships to the baseline primary and foreign key relationships.

8. A data migration apparatus based on business object mapping, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-5.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-5.

Images