Multi-source heterogeneous data standardization processing method and terminal based on meta-model adaptive matching

Abstract

The invention discloses a multi-source heterogeneous data standardization processing method and terminal based on meta-model adaptive matching, and relates to the technical field of data processing. Obtaining a multi-source heterogeneous data set containing original data units of different sources and structures, and recording data sources and collection scene attributes of the multi-source heterogeneous data set; performing pre-adaptive adjustment on a preset meta-model library based on the attributes to generate a pre-adaptive meta-model library; extracting multi-dimensional structural features from the original data unit, and carrying out multi-round adaptive matching on the multi-dimensional structural features and templates in the pre-adaptive meta-model library to generate a matching result and an optimization suggestion; calling a data conversion rule according to a matching result, and performing dynamic standardized conversion on the original data unit in combination with optimization suggestions to generate a standardized data unit and a conversion process record; and collecting data to establish a standardized data set, and performing association analysis to generate a processing result report and a meta-model library optimization scheme. According to the method, the accuracy and efficiency of standardized processing of the multi-source heterogeneous data are improved.

Description

Technical Field

[0001] This invention belongs to the field of multi-source heterogeneous data standardization processing technology, specifically involving a multi-source heterogeneous data standardization processing method and terminal based on meta-model adaptive matching. Background Technology

[0002] In today's digital age, data has become a crucial asset for enterprise and social development. With the widespread application of information technology, a large amount of multi-source heterogeneous data has been generated across different systems and business scenarios. These data sources are diverse, covering information systems of various business departments within an enterprise, data interfaces of external partners, and publicly available data on the Internet. The data structures also vary, including structured data (such as tabular data in relational databases), semi-structured data (such as data in XML and JSON formats), and unstructured data (such as text, images, and audio).

[0003] Existing methods for processing multi-source heterogeneous data mainly suffer from the following problems:

[0004] On the one hand, traditional data processing methods often use fixed meta-model templates to process all data, lacking targeted adaptation to different data sources and collection scenarios. This results in a mismatch between the meta-model and the actual data structure, making it impossible to accurately describe the characteristics and constraints of the data, thereby affecting the accuracy and effectiveness of data transformation.

[0005] On the other hand, the lack of an adaptive matching mechanism during the data transformation process makes it difficult to dynamically adjust the matching strategy according to the actual structural characteristics of complex and variable multi-source heterogeneous data. This can easily lead to matching errors or incomplete matching, resulting in inconsistent quality of the generated standardized data.

[0006] Furthermore, existing methods lack sufficient correlation analysis between transformation process records and standardized data during data processing, making it difficult to promptly identify problems and potential risks in the data processing process. They also make it difficult to optimize and improve the meta-model library based on actual conditions, which is not conducive to the continuous optimization and improvement of data processing. Summary of the Invention

[0007] To address the shortcomings of the existing technologies, the present invention aims to provide a method and terminal for standardizing multi-source heterogeneous data based on meta-model adaptive matching, which effectively avoids data processing errors caused by mismatch between the meta-model and the data.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] The present invention provides a method for standardizing multi-source heterogeneous data based on meta-model adaptive matching, comprising the following steps:

[0010] Acquire a multi-source heterogeneous data set, which contains raw data units from different data sources and with different data structures, and record the data source attributes and collection scenario attributes of each raw data unit;

[0011] Based on the data source attributes and collection scenario attributes of the multi-source heterogeneous data set, the pre-adapted metamodel library is adjusted to generate a pre-adapted metamodel library. The pre-adapted metamodel library contains initial metamodel templates designed for different data structure types. Each initial metamodel template contains data structure description information, data type constraint information, and data conversion rule information.

[0012] Data structure features are extracted from each original data unit in the multi-source heterogeneous data set to generate multi-dimensional structural features for each original data unit. Based on the multi-dimensional structural features, multi-round adaptive matching is performed with the meta-model templates in the pre-adapted meta-model library to generate meta-model matching results and matching optimization suggestions for each original data unit.

[0013] Based on the metamodel matching result, the data transformation rule information of the corresponding metamodel template is called, and the matching optimization suggestion is combined to perform dynamic standardization transformation on each original data unit, generating a standardized data unit and a transformation process record corresponding to each original data unit.

[0014] Collect standardized data units and transformation process records corresponding to all original data units, construct a standardized data set, perform correlation analysis on the standardized data set and transformation process records, and generate a report on the standardized processing results of multi-source heterogeneous data and an optimization scheme for the meta-model library.

[0015] Furthermore, this invention also provides a multi-source heterogeneous data standardization processing terminal based on meta-model adaptive matching, comprising:

[0016] One or more processors;

[0017] A machine-readable storage medium for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-described method for standardizing multi-source heterogeneous data based on meta-model adaptive matching.

[0019] In another aspect, the present invention also provides a computer program product, the computer program product including machine-executable instructions stored in a computer-readable storage medium, a processor of a computer device reading the machine-executable instructions from the computer-readable storage medium, and the processor executing the machine-executable instructions, causing the computer device to perform the above-described method for standardizing multi-source heterogeneous data based on meta-model adaptive matching.

[0020] The beneficial effects of this invention are:

[0021] This invention acquires and merges records of the data sources and acquisition scenarios of multi-source heterogeneous datasets. Based on these attributes, it pre-adapts and adjusts a pre-defined meta-model library to generate a pre-adapted meta-model library. This enhances the adaptability and relevance of the meta-model to multi-source heterogeneous data, effectively avoiding data processing errors caused by mismatch between the meta-model and the data. Multi-dimensional structural feature extraction and multi-round adaptive matching of the original data units dynamically adjust the matching strategy according to the actual characteristics of the data, improving the accuracy and completeness of meta-model matching. Based on the meta-model matching results and matching optimization suggestions, the original data units are dynamically standardized and transformed, generating standardized data units and transformation process records. All standardized data units and transformation process records are collected and correlated, generating a multi-source heterogeneous data standardization processing result report and a meta-model library optimization scheme. This achieves comprehensive evaluation and continuous optimization of the data processing process, improving the quality and efficiency of multi-source heterogeneous data standardization processing. Attached Figure Description

[0022] Figure 1 This is a schematic diagram illustrating the principle and flow of the multi-source heterogeneous data standardization processing method based on meta-model adaptive matching of the present invention.

[0023] Figure 2 This is a schematic diagram of exemplary hardware and software components of the multi-source heterogeneous data standardization processing terminal based on meta-model adaptive matching of the present invention. Detailed Implementation

[0024] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to embodiments and accompanying drawings. The content mentioned in the embodiments is not intended to limit the present invention.

[0025] Reference Figure 1 As shown, the present invention provides a method for standardizing multi-source heterogeneous data based on meta-model adaptive matching, comprising the following steps:

[0026] Step S110: Obtain a multi-source heterogeneous data set, which contains original data units from different data sources and with different data structures, and records the data source attributes and collection scenario attributes of each original data unit.

[0027] In this embodiment, taking the processing of multi-source business data of a certain enterprise as an example, the data sources include the enterprise's internal ERP system, CRM system, OA system, and industry data platform data provided by external partners. The original data unit can be an order data unit in the ERP system, a customer information data unit in the CRM system, an office process data unit in the OA system, and a market data unit in the industry data platform, etc. For each original data unit, its data source attribute is recorded. For example, the data source attribute of the ERP system can be marked as "ERP-Production Order", the CRM system as "CRM-Customer Management", the OA system as "OA-Process Approval", and the external industry data platform as "Industry-Market". The collection scenario attribute is recorded according to the business scenario at the time of data collection. For example, the collection scenario attribute of the order data unit in the ERP system is "Production Order Creation", the collection scenario attribute of the customer information data unit in the CRM system is "Customer Information Entry and Update", the collection scenario attribute of the office process data unit in the OA system is "Process Initiation and Approval", and the collection scenario attribute of the market data unit in the external industry data platform is "Market Monitoring".

[0028] Step S120: Based on the data source attributes and collection scenario attributes of the multi-source heterogeneous data set, perform metamodel pre-adaptation adjustment on the preset metamodel library to generate a pre-adapted metamodel library. The preset metamodel library contains initial metamodel templates designed for different data structure types. Each initial metamodel template contains data structure description information, data type constraint information, and data conversion rule information.

[0029] Specifically, step S120 is as follows:

[0030] Step S121: Extract the data source attributes of all original data units in the multi-source heterogeneous data set, classify and statistically analyze the number of original data units and the types of data structure differences corresponding to different data sources, and generate a data source structure distribution table;

[0031] In this embodiment, for the multi-source business data of the aforementioned enterprises, the data source attributes of each original data unit are extracted. For example, the number of original data units under the "ERP-Production Order" data source in the ERP system is counted. Assuming that there are multiple order-related data units under this data source, their number is counted. At the same time, the data structure difference types of these data units are analyzed. For example, some order data units contain fields such as order number, order date, customer number, product number, quantity, and unit price, with a relatively regular structure, while others may also contain some additional remarks fields or custom fields, resulting in structural differences. The above difference types are classified and statistically analyzed, such as differences in the number of fields, differences in the order of fields, and fields with similar meanings but different names. Then, the statistical results are organized into a data source structure distribution table, which contains information such as data source attributes, the number of original data units, and data structure difference types.

[0032] Step S122: Extract the collection scenario attributes of all original data units in the multi-source heterogeneous data set, analyze the variation law of the number of fields and the characteristics of field association in the original data units under different collection scenarios, and generate a scenario field feature table;

[0033] For the aforementioned enterprise data, we extract the collection scenario attributes of each raw data unit. For example, we analyze the raw data units under the "production order creation" collection scenario, count the number of fields in different order data units, and observe their changing patterns. For instance, some order data units have more fields and contain more detailed product information, while others have relatively fewer fields and only contain basic order information. At the same time, we analyze the characteristics of field association methods, such as the association method between the order number field and the customer number field and the product number field, whether it is through foreign key association or through business logic association, and the tightness of the association. We then organize the above analysis results into a scenario field feature table, which contains information such as collection scenario attributes, field number changing patterns, and field association method characteristics.

[0034] Step S123: Retrieve all initial metamodel templates in the preset metamodel library, and extract the field quantity range and field association type from the data structure description information of each initial metamodel template;

[0035] The preset metamodel library contains initial metamodel templates designed for different data structure types. For example, the initial metamodel template designed for order data structures may have a preset range for the number of fields in its data structure description information to cover the number of fields in common order data units; the field association type may include foreign key association, business logic association, etc. Now, all initial metamodel templates are retrieved from the preset metamodel library, and the range of the number of fields and the field association type in the data structure description information of each template are extracted. For example, the field range of a certain order metamodel template is [5, 15], and the field association type is foreign key association and business logic association.

[0036] Step S124: Compare the data structure difference type in the data source structure distribution table with the field association type of each initial metamodel template, and mark the initial metamodel templates whose field association methods do not match;

[0037] The data structure difference types of the original data units from different data sources, as statistically analyzed in the data source structure distribution table, are compared with the field association type of each initial metamodel template. For example, if the data structure difference type of an original data unit from a certain data source contains a specific field association type, but the field association type of an initial metamodel template does not contain this association type, then that initial metamodel template is marked as a template with a mismatch in field association type.

[0038] Step S125: Compare the variation pattern of the number of fields in the scene field feature table with the range of the number of fields in each initial metamodel template, and mark the initial metamodel templates whose field number ranges do not match.

[0039] The variation patterns of the number of fields in raw data units under different acquisition scenarios, as statistically analyzed in the scenario field feature table, are compared with the field number range of each initial metamodel template. For example, if the variation pattern of the number of fields in raw data units under a certain acquisition scenario shows that the number of fields often exceeds the field number range of a certain initial metamodel template, then that initial metamodel template is marked as a template with a mismatch in the field number range.

[0040] Step S126: For initial metamodel templates with mismatched field association methods, adjust the field association method type in their data structure description information so that the adjusted field association method type is adapted to the structural difference type of the corresponding data source.

[0041] For initial metamodel templates where the marked field association methods do not match, the field association method type in their data structure description information is adjusted according to the structural difference type of the corresponding data source. For example, if the original data unit under a certain data source has a time series-based field association method, but the corresponding initial metamodel template does not have this association method, then the field association method type of the template is adjusted to add a time series-based field association method type to make it compatible with the structural difference type of the data source.

[0042] Step S127: For initial metamodel templates with mismatched field quantity ranges, adjust the field quantity range in their data structure description information so that the adjusted field quantity range adapts to the field quantity variation pattern of the corresponding collection scenario.

[0043] For initial metamodel templates where the range of the number of marked fields does not match, the range of the number of fields in their data structure description information is adjusted according to the variation pattern of the number of fields in the corresponding data collection scenario. For example, if the number of fields in the original data unit in a certain data collection scenario often varies between [10, 20], while the range of the number of fields in the corresponding initial metamodel template is [5, 15], then the range of the number of fields in the template is adjusted to [8, 22] to adapt to the variation pattern of the number of fields in the data collection scenario.

[0044] Step S128: After the adjustment is completed, the adaptability of each adjusted metamodel template is verified. The original data units of the corresponding data source and collection scenario are selected to test the coverage of the data structure of the template.

[0045] After adjusting the metamodel template, select the original data units from the corresponding data sources and collection scenarios to verify the adaptability of the adjusted metamodel template. For example, for the adjusted order-type metamodel template, select the original data units from the "Production Order Creation" collection scenario in the ERP system, match the above data units with the template, and test the template's coverage of the data structure of the above data units. That is, check whether the template's fields, association methods, etc., can cover the data structure of these data units.

[0046] Step S129: Based on the adaptability verification results, fine-tune the field constraints and conversion rule priorities of the metamodel template until the template's coverage of the data structure in the corresponding data source and collection scenario reaches the preset deviation range.

[0047] Specifically, step S129 is as follows:

[0048] Step S1291: Extract the coverage of each metamodel template to the original data units in the corresponding data source and collection scenario during the adaptability verification process, and count the number of original data units not covered by the template and the reasons for not being covered;

[0049] During the compatibility verification process, the coverage of each adjusted metamodel template with the original data units in the corresponding data source and collection scenario is statistically analyzed. For example, the number of original data units whose data structures are not covered by the template is counted, and the reasons for the lack of coverage are analyzed. It may be that the field constraints of the template are too strict, resulting in some fields that conform to the business logic but have slightly different data formats not being covered, or that the priority of the conversion rules is set improperly, causing problems in the conversion process of some data units.

[0050] If the reason for the lack of coverage is that the field constraints of the template are too strict, then analyze the field attributes of the original data units that are not covered, adjust the data type range or format requirements in the template field constraints, and expand the range of compatibility of the constraints.

[0051] For example, the data type and format of these fields can be adjusted, and then the allowed range of data types or format requirements in the template field constraints can be modified. For instance, if the data type of a field in a certain original data unit is a special date format, but the template field constraints have strict requirements on the date format, allowing only one common date format, then the template field constraints can be adjusted to expand the allowed range of date formats to cover the special date format.

[0052] If the reason for the lack of coverage is that the priority setting of the template's transformation rules is improper, causing some original data units to fail to be transformed according to the rules, then the applicable scenarios of the transformation rules should be re-evaluated, and the priority order of different transformation rules should be adjusted so that rules with a wide range of applicable scenarios are executed first.

[0053] For example, if a certain transformation rule has a wide range of applicable scenarios, but has a low priority in the template, some data units may be converted by other rules with narrower applicable scenarios first, resulting in transformation problems. In this case, the priority of the transformation rule with a wide range of applicable scenarios will be increased so that it will be executed first.

[0054] Step S1292: Select the uncovered original data units and perform the adaptation test again. Count the number of uncovered original data units covered by the adjusted template and calculate the coverage degree.

[0055] After adjusting the field constraints or conversion rule priorities of the template, select the previously uncovered original data units and perform the compatibility test again. Count the number of uncovered original data units covered by the adjusted template, and then calculate the coverage degree. The coverage degree can be calculated by the ratio of the number of covered units to the total number of uncovered original data units.

[0056] If the coverage does not reach the preset deviation range, repeat the above steps of adjusting field constraints or adjusting conversion rule priority until the template's coverage of the original data units in the corresponding data source and collection scenario reaches the preset deviation range.

[0057] If the calculated coverage does not reach the preset deviation range, repeat the above steps of adjusting field constraints or conversion rule priorities, adjust the template again, and then select the uncovered original data units again for adaptation testing and calculate the coverage until the coverage reaches the preset deviation range.

[0058] Step S1293: Record the specific content of each adjustment, the change in coverage after the adjustment, and the information of the original data units used for testing, forming a template fine-tuning record;

[0059] After each adjustment to the template, record the specific details of the adjustment, such as which field constraints were adjusted and which conversion rules were prioritized. Also record the changes in coverage after the adjustment, as well as the information of the original data units used for testing, such as data source attributes and collection scenario attributes. Organize the above information into a template fine-tuning record.

[0060] Step S1294: Integrate all adjusted metamodel templates that have passed adaptability verification, build a pre-adapted metamodel library, and record the adjustment content of each metamodel template and the data source and collection scene information for adaptation;

[0061] Specifically, step S1294 is as follows:

[0062] Step S12941: Create the storage architecture of the pre-adapted metamodel library, and set up three functional areas: template storage area, adjustment record area, and adaptation information area. The template storage area is used to store the adjusted metamodel template, the adjustment record area is used to store the template adjustment content, and the adaptation information area is used to store the data source and collection scene information of the template adaptation.

[0063] Create a storage architecture for the pre-adapted metamodel library. The template storage area is used to store all adjusted metamodel templates that have passed adaptability verification. Each template has a unique identifier. The adjustment record area is used to store the adjustment content of each template, including the state before adjustment, the specific items adjusted, and the state after adjustment. The adaptation information area is used to store information such as the data source attributes and collection scenario attributes that each template is adapted to.

[0064] Step S12942: Upload each adjusted metamodel template that has passed the adaptability verification to the template storage area, and assign a unique template number to each metamodel template. The template number contains the encoded information of the adaptable data source identifier and the adaptable acquisition scenario identifier.

[0065] Each adjusted metamodel template that passes compatibility verification is uploaded to the template storage area, and a unique template number is assigned to each template. This template number contains coded information about the data source and the collection scenario. For example, for a metamodel template adapted to the "Production Order Creation" collection scenario of an ERP system, its template number can contain coded information for "ERP" and "Production Order Creation".

[0066] Step S12943: Compile the adjustment content of each metamodel template, including the template status before adjustment, the specific items to be adjusted, and the status after adjustment, into an adjustment record table, upload it to the adjustment record area, and establish the association between the adjustment record table and the corresponding metamodel template through the template number;

[0067] The adjustments to each metamodel template are compiled into an adjustment record table, including the template status before adjustment, such as field association type and field quantity range, the specific items adjusted, such as which field constraints and conversion rule priorities were adjusted, and the status after adjustment. The adjustment record table is then uploaded to the adjustment record area and associated with the corresponding metamodel template by template number for subsequent querying and management.

[0068] Step S12944: Compile the data source attributes, collection scenario attributes, and adaptation verification results of each metamodel template into an adaptation information table, upload it to the adaptation information area, and establish the association between the adaptation information table and the corresponding template through the template number. Confirm that the pre-adapted metamodel library has been built and generate a library construction completion report. The library construction completion report includes the number of templates in the pre-adapted metamodel library, the types of data sources covered, the types of collection scenarios covered, and a description of the library retrieval mechanism.

[0069] The data source attributes, collection scenario attributes, and adaptation verification results for each metamodel template are compiled into an adaptation information table, which is then uploaded to the adaptation information area. The adaptation information table is associated with the corresponding metamodel template using its template number. Finally, a library construction completion report is generated. This report includes the number of templates in the pre-adapted metamodel library, the types of data sources covered, the types of collection scenarios covered, and a description of the library's retrieval mechanism. For example, templates in the pre-adapted metamodel library can be retrieved using data source attributes, collection scenario attributes, template numbers, etc.

[0070] Step S130: Extract data structure features from each original data unit in the multi-source heterogeneous data set to generate multi-dimensional structural features for each original data unit. Perform multi-round adaptive matching based on the multi-dimensional structural features and the meta-model templates in the pre-adapted meta-model library to generate meta-model matching results and matching optimization suggestions for each original data unit; specifically as follows:

[0071] Step S131: Select a single original data unit from the multi-source heterogeneous data set, perform field hierarchical parsing on the original data unit, determine the hierarchical affiliation and depth of each field, and generate a field hierarchy structure table.

[0072] Taking an order data unit in an enterprise ERP system as an example, this original data unit undergoes hierarchical parsing of its fields. For instance, this order data unit contains order header information and order line information. Fields in the order header information, such as order number, order date, and customer number, belong to the first level. Fields in the order line information, such as product number, quantity, and unit price, belong to the second level. If the order line information also contains detailed product specification fields, such as product size and color, it belongs to the third level. After determining the level and depth of each field, a field hierarchy structure table is generated, containing information such as field name, level, and depth.

[0073] Step S132: Analyze the reference relationships and dependencies between different fields in the original data unit, count the number of field associations and the length of the association path, and generate a field association table;

[0074] Analyze the reference and dependency relationships between different fields in this order data unit. For example, the order number field is referenced by multiple fields in the order line information, and the customer number field is dependent on the customer name field in the order header information because the value of the customer name field depends on the customer information corresponding to the customer number field. Count the number of field associations, i.e., how many pairs of fields have references or dependencies, and the length of the association path. For example, the association path length from the order number field to a certain order line information field is 1, the association path length from the customer number field to the customer name field is 1, and the association path length from the order number field to a certain product specification information field is 2 (passing through the order line information field). Organize the above statistical results into a field association table, which includes information such as field name, associated field name, association type (reference or dependency), and association path length.

[0075] Step S133: Extract the data type and data format of each field in the original data unit, count the proportion of different data types in all fields, and generate a data type distribution table;

[0076] Extract the data type and format of each field in the order data unit. For example, the order number field is a string with the format "ORD-0001"; the order date field is a date with the format "YYYY-MM-DD"; the customer number field is a string with the format "CUST-0001"; the product number field is a string with the format "PROD-0001"; the quantity field is an integer; and the unit price field is a floating-point number. Calculate the percentage of each data type among all fields, such as the percentage of string fields, and the percentages of date, integer, and floating-point number fields. Then generate a data type distribution table containing information such as data type, number of fields, and percentage.

[0077] Step S134: Based on the field hierarchy structure table, field association table, and data type distribution table, construct the multi-dimensional structural features of the original data unit. The multi-dimensional structural features include hierarchical features, association features, and type features.

[0078] Specifically, step S134 is as follows:

[0079] Step S1341: Extract the level depth of each field from the field hierarchy structure table, calculate the average and maximum level depth of all fields, and use the average level depth, the maximum level depth, and the distribution of the number of fields at different level depths as the constituent elements of the level feature.

[0080] Extract the hierarchy depth of each field from the field hierarchy structure table. For example, the hierarchy depths of fields in this order data unit are 1, 1, 1, 2, 2, 3, etc. Calculate the average of all field hierarchy depths, which is the sum of all hierarchy depths divided by the number of fields, and the maximum value, which is the largest value among the hierarchy depths. Simultaneously, calculate the distribution of the number of fields at different hierarchy depths, such as the number of fields with a hierarchy depth of 1, the number of fields with a hierarchy depth of 2, the number of fields with a hierarchy depth of 3, etc. Use these as the constituent elements of the hierarchy feature.

[0081] Step S1342: Extract the number of field associations and the length of the association path from the field association table, calculate the average number of associations for each field and the median of all association path lengths, and use the average number of field associations, the median of the association path length, and the proportion of fields with more than a preset number of associations as the constituent elements of the association feature.

[0082] Extract the number of field associations (i.e., the number of association pairs between all fields) and the length of the association paths from the field relationship table. Then, calculate the average number of associations for each field (total number of associations divided by the number of fields) and the median of all association path lengths (the value in the middle after sorting all association path lengths in ascending order). Simultaneously, calculate the percentage of fields with more than a preset number of associations. Use these as the constituent elements of the association feature.

[0083] Step S1343: Extract the field proportion of different data types from the data type distribution table, filter data types whose field proportion exceeds the preset proportion threshold to form a list of main data types, and use the list of main data types, the proportion value of each data type in the list of main data types, and the number of data types as the constituent elements of the type feature;

[0084] Extract the field percentages of different data types from the data type distribution table. For example, string type fields account for 60%, date type fields account for 10%, integer type fields account for 20%, and floating-point type fields account for 10%. Filter data types whose field percentages exceed a preset percentage threshold. Assuming the preset percentage threshold is 15%, then string type and integer type fields exceed the threshold, forming a list of main data types: string and integer. Then, use this list of main data types, the percentage values ​​of each data type (60%, 20%), and the number of data types (2) as the constituent elements of the type feature.

[0085] Step S1344: Perform unified format processing on each component of hierarchical features, association features, and type features, and describe each element using the same data representation format;

[0086] The constituent elements of hierarchical features, related features, and type features are processed in a unified format, such as using text descriptions or specific encoding formats, to ensure that the descriptions of each element are consistent for subsequent comparison and processing.

[0087] Step S1345: Integrate the processed hierarchical features, association features, and type features to form the multi-dimensional structural features of the original data unit, and assign a unique identifier to each dimension feature;

[0088] The processed hierarchical features, association features, and type features are integrated to form the multi-dimensional structural features of the original data unit. For example, the hierarchical features are identified as “Level_Feature”, the association features as “Association_Feature”, and the type features as “Type_Feature”. These features are then organized according to a certain structure to form multi-dimensional structural features.

[0089] Step S135: Retrieve all metamodel templates from the pre-adapted metamodel library, and extract the hierarchical standards, association standards, and type standards corresponding to the data structure description information of each metamodel template;

[0090] Retrieve all metamodel templates from the pre-adapted metamodel library, such as metamodel templates for order data and customer data. Extract the hierarchical, association, and type standards corresponding to the data structure description information of each metamodel template. For example, the hierarchical standards for the order metamodel template include the allowed range of hierarchical depth and the required number of fields at different levels; the association standards include the type of field association and the limit on the length of the association path; and the type standards include the allowed data types and the required proportion of each data type.

[0091] Step S136: In the first round of matching, the hierarchical features of the multi-dimensional structural features are compared with the hierarchical standards of each meta-model template. The hierarchical fit is generated by comparing the number of corresponding field hierarchies with the degree of fit of the hierarchical depth. The candidate template set that meets the first preset threshold is then selected.

[0092] Specifically, step S136 is as follows:

[0093] Step S1361: Extract the number of corresponding field levels and the distribution of level depth from the hierarchical features of the multi-dimensional structural features, and extract the number of standard field levels and the range of standard level depth from the hierarchical standards of each meta-model template;

[0094] Extract the number of fields corresponding to different levels from the hierarchical features of the multi-dimensional structural features, i.e., the number of fields at different levels, and the distribution of the number of fields at each level depth. Extract the standard number of field levels from the hierarchical standards of each metamodel template, i.e., the number of fields at different levels required by the template, and the standard range of level depth, i.e., the range of level depth allowed by the template.

[0095] Step S1362: Compare the number of corresponding field levels in the original data unit with the number of standard field levels in the meta-model template, and calculate the number matching ratio. The number matching ratio is obtained by comparing the number of corresponding field levels with the number of standard field levels.

[0096] The number of fields corresponding to the original data unit's field hierarchy is compared with the number of fields corresponding to the standard field hierarchy in the metamodel template. For example, in the original data unit's field hierarchy, the first level has 3 fields, the second level has 5 fields, and the third level has 2 fields; while in the standard field hierarchy of the metamodel template, the first level requires 3-5 fields, the second level requires 4-6 fields, and the third level requires 1-3 fields. The matching ratio is calculated, that is, the degree of matching between the number of fields at each level and the number of fields at the standard field hierarchy is calculated separately, and then the overall matching ratio is obtained.

[0097] Step S1363: Compare the hierarchical depth distribution of the original data units with the standard hierarchical depth range of the meta-model template, count the number of hierarchical depths within the standard hierarchical depth range, and calculate the depth matching ratio. The depth matching ratio is obtained by comparing the number of hierarchical depths within the standard range with the total number of hierarchical depths.

[0098] The layer depth distribution of the original data units is compared with the standard layer depth range of the metamodel template. For example, if the original data units have layer depths of 1, 2, and 3, while the standard layer depth range of the metamodel template is 1-3, then the number of layer depths within this range is counted, i.e., the total number of layer depths. Since they are all within the range, the depth matching ratio is 100%. If the original data units have a layer depth of 4, and the standard layer depth range is 1-3, then the number of layer depths within the range is the total number of layer depths minus the number of layer depths with a depth of 4, and then the depth matching ratio is calculated.

[0099] Step S1364: Calculate the level fit degree according to the quantity matching ratio and the depth matching ratio, based on the preset weights. The level fit degree is obtained by multiplying the quantity matching ratio by the first weight and the depth matching ratio by the second weight. The sum of the first weight and the second weight is a fixed value.

[0100] Based on the calculated quantity matching ratio and depth matching ratio, the hierarchical adaptation degree is calculated according to the preset weights. For example, if the first weight is 0.6, the second weight is 0.4, the quantity matching ratio is 0.8, and the depth matching ratio is 0.9, then the hierarchical adaptation degree is 0.8×0.6+0.9×0.4=0.48+0.36=0.84.

[0101] Step S1365: Compare the hierarchical fit of each metamodel template with the first preset threshold, select metamodel templates with hierarchical fit greater than or equal to the first preset threshold, and form a candidate template set.

[0102] The hierarchical fit of each metamodel template is compared with a first preset threshold. Assuming the first preset threshold is 0.7, metamodel templates with a hierarchical fit greater than or equal to 0.7 are selected to form a candidate template set.

[0103] Step S1366: Record the hierarchical fit value of each template in the candidate template set and the quantity matching ratio and depth matching ratio used in the calculation process to form the first round of matching records;

[0104] Record the hierarchical fit value of each template in the candidate template set, as well as the quantity matching ratio and depth matching ratio used in the calculation process, to form the first round of matching records for subsequent viewing and analysis.

[0105] Step S137: In the second round of matching, the association features of the multi-dimensional structural features are compared with the association criteria of each template in the candidate template set. The association fit is generated by comparing the matching degree of the number of field associations and the fit of the association path length. The secondary candidate template set that meets the second preset threshold is then selected.

[0106] Specifically, step S137 is as follows:

[0107] Step S1371: Extract the number of field associations and the length of association paths from the association features of the multi-dimensional structural features, and extract the number of standard field associations and the range of standard association path lengths from the association criteria of each template in the candidate template set;

[0108] Extract the number of field associations from the association features of the multi-dimensional structural features, i.e., the number of association pairs between fields in the original data unit, and the association path length, i.e., the length of each association path. Extract the number of standard field associations from the association criteria of each template in the candidate template set, i.e., the number of field association pairs required by the template, and the standard association path length range, i.e., the range of association path lengths allowed by the template.

[0109] Step S1372: Compare the number of field associations in the original data unit with the number of standard field associations in each template in the candidate template set, and calculate the number matching degree. The number matching degree is obtained by comparing the number of field associations with the number of standard field associations.

[0110] The number of field associations in the original data unit is compared with the standard number of field associations for each template in the candidate template set. For example, if the number of field associations in the original data unit is 10, and the standard number of field associations for a certain template is 8-12, then the number matching degree is 1 (because 10 is within the range of 8-12). If the number of field associations in the original data unit is 15, and the standard number of field associations for the template is 8-12, then the number matching degree is 12 / 15=0.8 (assuming that the part exceeding the standard number is calculated proportionally).

[0111] Step S1373: Compare the association path length of the original data unit with the standard association path length range of each template in the candidate template set, count the number of association path lengths within the standard association path length range, and calculate the length fit. The length fit is obtained by comparing the number of association path lengths within the standard range with the total number of association path lengths.

[0112] The association path length of the original data unit is compared with the standard association path length range of each template in the candidate template set. For example, the association path length of the original data unit is 1, 2, or 3, while the standard association path length range of a certain template is 1-2. Then, the number of association path lengths within this range is counted. Assuming there are a total of 10 association paths, and 8 of them have a length in the range of 1-2, then the length matching degree is 8 / 10 = 0.8.

[0113] Step S1374: Calculate the association fit degree according to the quantity matching degree and length fit degree with preset weights. The association fit degree is obtained by multiplying the quantity matching degree by the third weight and the length fit degree by the fourth weight. The sum of the third weight and the fourth weight is a fixed value.

[0114] The association fit is calculated based on the quantity matching degree and length matching degree according to the preset weights. For example, if the third weight is 0.5, the fourth weight is 0.5, the quantity matching degree is 0.8, and the length matching degree is 0.9, then the association fit is 0.8×0.5+0.9×0.5=0.4+0.45=0.85.

[0115] Step S1375: Compare the correlation fit of each candidate template with the second preset threshold, select candidate templates with a correlation fit greater than or equal to the second preset threshold, and form a secondary candidate template set.

[0116] The correlation fit of each candidate template is compared with the second preset threshold. Assuming the second preset threshold is 0.75, candidate templates with a correlation fit greater than or equal to 0.75 are selected to form a secondary candidate template set.

[0117] Step S1376: Record the correlation fit value of each template in the secondary candidate template set and the quantity matching degree and length fit degree used in the calculation process to form the second round of matching records;

[0118] Record the correlation fit value of each template in the secondary candidate template set, as well as the quantity matching degree and length fit degree used in the calculation process, to form the second round of matching records.

[0119] Step S138: In the third round of matching, the type features of the multi-dimensional structural features are compared with the type standards of each template in the secondary candidate template set. The type fit is generated by comparing the data type proportion matching degree and the data format conformity degree. The template with the highest type fit is selected as the initial matching template.

[0120] Specifically, step S138 is as follows:

[0121] Step S1381: Extract the data type ratio and data format requirements from the type features of the multi-dimensional structural features, and extract the standard data type ratio and standard data format requirements from the type standards of each template in the secondary candidate template set;

[0122] Extract the data type proportion from the type features of the multi-dimensional structural features, i.e., the proportion of each data type in the original data unit, and the data format requirements, i.e., the data format requirements of each field in the original data unit. Extract the standard data type proportion from the type criteria of each template in the secondary candidate template set, i.e., the proportion range of each data type required by the template, and the standard data format requirements, i.e., the data formats allowed by the template.

[0123] Step S1382: Compare the data type proportion of the original data unit with the standard data type proportion of each template in the secondary candidate template set, and calculate the type proportion matching degree. The type proportion matching degree is obtained by comparing the data type proportion with the standard data type proportion.

[0124] The data type proportions of the original data units are compared with the standard data type proportions of each template in the secondary candidate template set. For example, if the proportion of string types in the original data units is 60% and the proportion of integer types is 20%, while the standard data type proportions of a certain template are 50%-70% string types and 15%-25% integer types, then the type proportion matching degree (60% within the 50%-70% range, 20% within the 15%-25% range) can be calculated as 1 (assuming both are within the range, the matching degree is 1). If the proportion of string types in the original data units is 75%, exceeding the standard data type proportion range of the template (50%-70%), then the type proportion matching degree is 70% / 75% = 0.93 (assuming the excess portion is calculated proportionally).

[0125] Step S1383: Compare the data format requirements of the original data unit with the standard data format requirements of each template in the secondary candidate template set, and calculate the format matching degree. The format matching degree is obtained by comparing the data format requirements with the standard data format requirements.

[0126] The data format requirements of the original data unit are compared with the standard data format requirements of each template in the secondary candidate template set. For example, if the data format of the order number field in the original data unit is "ORD-0001", while the standard data format requirement of a certain template is "ORD-[0-9]{4}", then the format matching degree is 1 (because it meets the requirements of this regular expression). If the data format of a field in the original data unit is "0001-ORD", which does not match the standard data format requirements of the template, then the format matching degree is 0.

[0127] Step S1384: Calculate the type fit degree according to the type proportion matching degree and the format matching degree, based on the preset weights. The type fit degree is obtained by multiplying the type proportion matching degree by the fifth weight and the format matching degree by the sixth weight. The sum of the fifth weight and the sixth weight is a fixed value.

[0128] The type fit is calculated based on the type proportion matching degree and the format matching degree, according to the preset weights. For example, if the fifth weight is 0.6, the sixth weight is 0.4, the type proportion matching degree is 0.9, and the format matching degree is 0.8, then the type fit is 0.9×0.6+0.8×0.4=0.54+0.32=0.86.

[0129] Step S1385: Compare the type fit of each template in the secondary candidate template set, and select the template with the highest type fit as the initial matching template;

[0130] The type fit of each template in the secondary candidate template set is compared, and the template with the highest type fit is selected as the initial matching template. For example, if the type fit of template A is 0.86, the type fit of template B is 0.82, and the type fit of template C is 0.88, then template C is selected as the initial matching template.

[0131] Step S139: Analyze the adaptation deviation between the preliminary matching template and the multi-dimensional structural features. If the deviation is within a preset deviation range, the preliminary matching template is determined as the final matching template, and a meta-model matching result containing the final matching template identifier and the adaptation degree of each round is generated. If the deviation exceeds the preset deviation range, matching optimization suggestions are generated based on the deviation content. The matching optimization suggestions include the template standard items that need to be adjusted and the adjustment direction. The preliminary matching template and the matching optimization suggestions are used together as the meta-model matching result.

[0132] Analyze the adaptation deviation between the initial matching template (e.g., template C) and multi-dimensional structural features, such as checking the differences between hierarchical features, association features, type features, and the standards of template C. If the deviation is within a preset deviation range, for example, if the deviation value is within the allowable error range, then template C is determined as the final matching template, and a metamodel matching result is generated, containing the identifier of template C and the adaptation degree of each round (hierarchical adaptation degree, association adaptation degree, type adaptation degree). If the deviation exceeds the preset deviation range, for example, if the hierarchical depth distribution differs significantly from the standard hierarchical depth range of template C, then matching optimization suggestions are generated based on the deviation content. It is suggested to adjust the standard hierarchical depth range in the hierarchical standard of template C, and the adjustment direction is to expand the range to cover the hierarchical depth distribution of the original data units. Then, template C and the matching optimization suggestions are used together as the metamodel matching result.

[0133] Step S140: Based on the metamodel matching result, call the data transformation rule information of the corresponding metamodel template, and combine it with the matching optimization suggestions to dynamically standardize each original data unit, generating a standardized data unit and a transformation process record corresponding to each original data unit; specifically as follows:

[0134] Step S141: Extract the final matching template from the meta-model matching result, and retrieve the data transformation rule information contained in the final matching template. The data transformation rule information includes the transformation operation sequence, field mapping relationship and content adjustment method.

[0135] Extract the final matching template from the metamodel matching results, such as template C, and then retrieve the data transformation rule information contained in the template. This data transformation rule information includes a sequence of transformation operations, such as first performing field mapping, then data type conversion, and finally adjusting field associations; field mapping relationships, such as mapping the original field name "Order Number" to the standard field name "OrderNo", and mapping "Customer Number" to "CustomerNo", etc.; and content adjustment methods, such as converting the original data format "ORD-0001" of the Order Number field to the standard format "Order-0001", and converting the date format "YYYY / MM / DD" to "YYYY-MM-DD", etc.

[0136] Step S142: If the meta-model matching result contains matching optimization suggestions, then adjust the sequence order of transformation operations or the priority of field mapping relationships in the data transformation rule information according to the matching optimization suggestions to generate optimized data transformation rule information;

[0137] Specifically, step 142) includes:

[0138] Step S1421: parse the matching optimization suggestion, determine the template standard item that needs to be adjusted in the matching optimization suggestion, and determine whether the component of the transformation rule corresponding to the template standard item is a transformation operation sequence or a field mapping relationship;

[0139] The analysis process involves resolving matching optimization suggestions. For example, if the suggested template standard item for adjustment is the standard hierarchy depth range in the hierarchy standard, then the components of the transformation rule corresponding to that standard item are determined. If the adjustment of the hierarchy standard will affect the order of the transformation operation sequence, such as requiring the adjustment of the hierarchy affiliation of fields before field mapping, then the components of the transformation rule are the transformation operation sequence. If the adjustment of the hierarchy standard will affect the priority of field mapping relationships, such as changes in the hierarchy affiliation of certain fields causing changes in their mapping priority, then the components of the transformation rule are the field mapping relationships.

[0140] If the adjustment needs to be made to the sequence of transformation operations, then analyze the adjustment direction in the matching optimization suggestions. If the adjustment direction is to increase the execution priority of the target class transformation operation, then move the target class transformation operation to the beginning of the transformation operation sequence. If the adjustment direction is to split a transformation operation, then split the transformation operation into multiple sub-operations and insert them into the transformation operation sequence in logical order.

[0141] If the adjustment requires changing the sequence of transformation operations, analyze the direction of adjustment in the matching optimization suggestions. For example, if the adjustment direction is to increase the execution priority of field level adjustment operations because the hierarchical depth distribution of the original data units differs from the template standard, the hierarchical affiliation of fields needs to be adjusted first, then move the field level adjustment operation to the beginning of the transformation operation sequence. For example, if the original transformation operation sequence was field mapping, data type conversion, and field association adjustment, it can be adjusted to field level adjustment, field mapping, data type conversion, and field association adjustment. If the adjustment direction is to split the data type conversion operation because the data type of the original data units is complex and requires more detailed conversion, then split the data type conversion operation into sub-operations such as integer type conversion, floating-point number type conversion, and string type conversion, and insert them into the transformation operation sequence in logical order.

[0142] If the priority of field mapping relationships needs to be adjusted, the adjustment basis in the matching optimization suggestions is analyzed. If the adjustment basis is the semantic relevance of the fields, the preset semantic relevance rule library is called to score the fields, and the field mapping relationships are sorted in descending order according to the score results to determine the execution order. If the adjustment basis is the proportion of field data volume, the field mapping relationships are sorted in descending order directly according to the value of the proportion of each field data volume to determine the execution order.

[0143] If the adjustment requires prioritizing field mapping relationships, analyze the adjustment criteria in the matching optimization suggestions. If the criterion is the semantic relevance of the fields, then call the preset semantic relevance rule library to score the fields, such as scoring based on semantic similarity, business logic relevance, etc., and then sort the field mapping relationships in descending order based on the scoring results, with higher-scoring field mapping relationships executed first. If the criterion is the percentage of data volume for each field, then directly sort the field mapping relationships in descending order based on the percentage of data volume for each field, with field mapping relationships with higher percentages of data volume executed first.

[0144] Step S1422: Simulate the optimized transformation operation sequence and field mapping relationship, select some fields of the original data unit for trial transformation, and observe whether new adaptation deviations occur during the trial transformation process;

[0145] After adjusting the data transformation rules, simulate the optimized transformation operation sequence and field mapping relationships. Select some fields from the original data units, such as order number, customer number, and product number, and perform a trial transformation. Observe whether any new adaptation deviations occur during the trial transformation, such as whether the field mapping is correct, whether the data type conversion is successful, and whether the field association adjustments meet the requirements.

[0146] If no new adaptation deviations occur, the optimized data conversion rule information is deemed valid; if new adaptation deviations occur, the conversion operation sequence or field mapping relationship is fine-tuned according to the deviation situation, and the conversion is performed again until no new adaptation deviations occur.

[0147] If no new adaptation deviations occur during the trial conversion, the optimized data conversion rules are deemed valid. If new adaptation deviations occur, such as duplicate field mappings after adjusting field mapping relationships, the conversion operation sequence or field mapping relationships are fine-tuned based on the deviation, for example, by adjusting the priority of field mapping relationships. The trial conversion is then performed again until no new adaptation deviations occur.

[0148] Step S1423: Record the differences in data conversion rule information before and after adjustment, the deviations during the trial conversion process, and the final optimization results to form a conversion rule optimization record, which is a component of the conversion process record;

[0149] Record the differences in data transformation rules before and after the adjustment, such as changes in the transformation operation sequence and changes in the priority of field mapping relationships; record the deviations during the trial transformation process, such as new deviations that occur and their solutions; record the final optimization result, i.e., the optimized data transformation rule information. Organize the above information into a transformation rule optimization record as a component of the transformation process record.

[0150] Step S143: According to the conversion operation sequence in the optimized data conversion rule information, perform mapping processing on the fields of the original data unit, replace the original field names with standard field names according to the field mapping relationship, and record the field mapping correspondence;

[0151] Following the transformation operation sequence in the optimized data transformation rules, the fields of the original data units are first mapped. For example, based on the field mapping relationship, the original field name "Order Number" is replaced with the standard field name "OrderNo", "Customer Number" is replaced with "CustomerNo", and "Product Number" is replaced with "ProductNo", etc. The field mapping correspondence is recorded to form a field mapping correspondence table, which contains information such as the original field name, the standard field name, and the mapping time.

[0152] Step S144: Convert the data type of each field. Convert the original data type to the standard data type according to the type conversion requirements in the data conversion rule information. If a data format mismatch occurs during the conversion process, correct the data format according to the content adjustment method, and record the details of data type conversion and format correction.

[0153] After completing the field mapping process, the data type of each field is converted. For example, the quantity field in the original data unit is of integer data type, which is converted to the standard data type "int" according to the type conversion requirements; the unit price field is of floating-point data type, which is converted to the standard data type "float". If data format mismatch occurs during the conversion process, such as the original data format of the order date field being "YYYY / MM / DD" while the standard data format is "YYYY-MM-DD", then according to the content adjustment method, " / " is replaced with "-" to correct the data format. Details of data type conversion and format correction are recorded to form a data type conversion and format correction details table, which includes information such as field name, original data type, standard data type, original data format, standard data format, and conversion / correction time.

[0154] Step S145: Adjust the field associations of the original data units after data type conversion. According to the association standards in the data conversion rule information, adjust the reference and dependency relationships between fields to make the field associations conform to the association standards, and record the association adjustment steps.

[0155] Specifically, step S145 is as follows:

[0156] Step S1451: Extract field reference relationship specifications and field dependency relationship specifications from the association relationship standards of data transformation rule information;

[0157] Extract field reference relationship specifications from the association standards of data transformation rule information, such as which fields can reference which fields and how they are referenced; and field dependency relationship specifications, such as which fields depend on which fields and what the conditions for dependency are.

[0158] Step S1452: Analyze the current reference relationships and current dependencies between fields in the original data unit after the data type conversion is completed, and generate the current association table;

[0159] Analyze the current reference and dependency relationships between fields in the original data unit after data type conversion. For example, the OrderNo field is referenced by the ProductNo and Quantity fields, and the CustomerNo field has a dependency relationship with the CustomerName field. Generate a table of current relationships, which contains information such as field name, related field name, relationship type (reference or dependency), and relationship path.

[0160] Step S1453: Compare the current reference relationships in the current relationship table with the field reference relationship specifications of the relationship standard, and mark the reference relationship entries that do not conform to the specifications;

[0161] The current reference relationship in the current relationship table is compared with the field reference relationship specification of the relationship standard. For example, if the specification requires that the OrderNo field can only be referenced by a specific field, and the OrderNo field in the current reference relationship is referenced by an unauthorized field, then the reference relationship entry is marked as an entry that does not conform to the specification.

[0162] Step S1454: Compare the current dependencies in the current association table with the field dependency specification of the association standard, and mark the dependency entries that do not conform to the specification;

[0163] The current dependencies in the current relationship table are compared with the field dependency specifications of the relationship standard. For example, if the specification requires that the CustomerNo field can only depend on a specific field, and the CustomerNo field in the current dependency depends on an unallowed field, then the dependency entry is marked as an entry that does not conform to the specification.

[0164] Step S1455: For the marked reference relationship entries that do not conform to the specification, adjust the field reference direction or reference object according to the field reference relationship specification so that the adjusted reference relationship conforms to the specification;

[0165] For entries marked as non-compliant with the field reference relationship specification, adjustments should be made according to the specification. For example, if an entry is non-compliant because the OrderNo field is referenced by an unauthorized field, the reference object of that field should be adjusted to reference an authorized field, or the reference direction should be adjusted to conform to the specification.

[0166] Step S1456: For the marked dependency entries that do not conform to the specification, adjust the field dependency order or dependency conditions according to the field dependency specification so that the adjusted dependency conforms to the specification;

[0167] For dependency entries that do not conform to the specification, adjustments should be made according to the field dependency specification. For example, if a dependency entry does not conform to the specification because the CustomerNo field depends on an unallowed field, then the dependency order of that field should be adjusted to depend on an allowed field, or the dependency condition should be adjusted to conform to the specification.

[0168] Step S1457: Regenerate the adjusted relationship table, compare the adjusted relationship table with the relationship standard, and confirm that all field relationships conform to the specification;

[0169] After completing all relationship adjustments, regenerate the adjusted relationship table. Then, compare the adjusted relationship table with the relationship standard to confirm that all field relationships conform to the specification. If there are still entries that do not conform to the specification, continue adjusting until all field relationships conform to the specification.

[0170] Step S1458: Record the process of marking items that do not conform to the specifications, the specific operations of adjusting reference relationships and dependencies, and the differences in the relationship tables before and after adjustment, forming a relationship adjustment step record;

[0171] Record the process of marking entries that do not conform to the specifications, including the basis for marking and the number of entries marked; record the specific operations for adjusting reference relationships and dependencies, such as which fields were adjusted and the order of dependencies; record the differences in the relationship table before and after the adjustment, such as which entries changed. Organize the above information into a relationship adjustment step record.

[0172] Step S146: After completing all transformation operations, reorganize all processed fields according to the data structure description information of the final matching template to form standardized data units, and record the field composition and data content of the standardized data units;

[0173] After completing all transformation operations, reorganize all processed fields according to the data structure description information of the final matching template (such as template C). For example, according to the requirements of template C, organize the OrderNo, CustomerNo, ProductNo, Quantity, and Price fields in a certain order and structure to form standardized data units. Record the field composition of the standardized data unit, i.e., which fields it contains, and the data content, i.e., the specific value of each field, to form a standardized data unit record.

[0174] Step S147: Integrate the field mapping correspondence, data type conversion and format correction details, association adjustment steps and standardized data unit composition information to generate a conversion process record for the original data unit;

[0175] The information, including the field mapping correspondence table, data type conversion and format correction details table, relationship adjustment step record, and standardized data unit record, is integrated to generate a conversion process record for the original data unit. The record contains information about the entire conversion process of the original data unit for subsequent querying and analysis.

[0176] Step S150: Collect standardized data units and transformation process records corresponding to all original data units, construct a standardized data set, perform correlation analysis on the standardized data set and transformation process records, and generate a report on the standardization processing results of multi-source heterogeneous data and an optimization scheme for the meta-model library; specifically as follows:

[0177] Step S151: Traverse all the standardized data units and transformation process records corresponding to the original data units, collect the complete data content of each standardized data unit and the corresponding transformation process record, and establish a one-to-one correspondence between the standardized data units and the transformation process records;

[0178] This process iterates through all the standardized data units and transformation process records corresponding to the enterprise's original data units, such as those from ERP systems, CRM systems, OA systems, and external industry data platforms. It collects the complete data content of each standardized data unit; for example, the standardized data content of an order data unit includes the specific values ​​of fields such as OrderNo, CustomerNo, ProductNo, Quantity, and Price, as well as the corresponding transformation process records. Then, it establishes a one-to-one correspondence between standardized data units and transformation process records, for example, by associating them using a unique identifier for the data unit.

[0179] Step S152: Group all standardized data units according to the data source attributes of the corresponding original data units to form standardized data groups divided by data source, and count the number of standardized data units and the conversion completion rate for each data source;

[0180] All standardized data units are grouped according to the data source attributes of their corresponding original data units, for example, into ERP system data groups, CRM system data groups, OA system data groups, and external industry data platform data groups. The number of standardized data units corresponding to each data source is counted, i.e., how many standardized data units are contained in each data group, and the conversion completion rate is calculated, i.e., the ratio of the number of successfully converted standardized data units to the number of original data units under that data source.

[0181] Step S153: Perform field consistency analysis on the standardized data units in each standardized data group, compare the data content format of the same standard fields in different standardized data units under the same data source, and count the proportion of fields with uniform format.

[0182] Perform field consistency analysis on the standardized data units within each standardized data group. For example, for all standardized order data units in an ERP system data group, compare the data content format of the same standardized fields (such as OrderNo, CustomerNo, ProductNo, etc.). Calculate the percentage of fields with uniform formatting, i.e., the proportion of standardized fields with uniform data content format within the data group to the total number of standardized fields.

[0183] Step S154: Analyze all conversion process records, extract records of field mapping anomalies, data type conversion failures, and difficulties in adjusting association relationships that occurred during the conversion process, and count the number of occurrences of each type of anomaly and the corresponding data source attributes and meta-model template types;

[0184] Analyze all conversion process records and extract those exhibiting field mapping anomalies, data type conversion failures, or difficulties in adjusting relationships. For example, field mapping anomalies might occur when the original field cannot find a corresponding standard field; data type conversion failures might occur when the original data type cannot be converted to a standard data type; and difficulties in adjusting relationships might occur when field relationships do not conform to the relationship standards. Count the frequency of each type of anomaly, along with the corresponding data source attributes (e.g., ERP system, CRM system) and metamodel template types (e.g., order template, customer template).

[0185] Step S155: Based on the above statistics of the number of standardized data units, conversion completion rate, proportion of fields with unified format, and occurrence of various anomalies, generate a multi-source heterogeneous data standardization processing result report. The processing result report includes the standardization processing effectiveness of each data source, the main anomaly types in the conversion process, and the standardized data quality assessment.

[0186] Based on the statistical data of the number of standardized data units, conversion completion rate, percentage of fields with uniform format, and occurrence of various anomalies, a report on the standardization processing results of multi-source heterogeneous data is generated. The report includes the standardization processing effectiveness of each data source; for example, the data conversion completion rate for the ERP system is 95%, and the percentage of fields with uniform format is 90%. It also covers the main types of anomalies during the conversion process, such as the most frequent occurrence of field mapping anomalies, primarily concentrated in the CRM system data source; and a standardized data quality assessment, for example, the overall data quality is good, but some data sources still have room for optimization.

[0187] Step S156: Based on the anomaly records and standardized data quality assessment results during the conversion process, locate the metamodel template problem that caused the anomaly. If the anomaly is due to the imperfect conversion rules of the template, generate optimization suggestions for the template conversion rules; if the anomaly is due to the mismatch of the template's structural standards, generate suggestions for adjusting the template's structural standards.

[0188] Specifically, step S156 includes:

[0189] Step S1561: Classify and organize the abnormal records in the conversion process, and divide the abnormal records into three categories: field mapping abnormalities, data type conversion abnormalities, and relationship adjustment abnormalities. Count the meta-model template identifiers and corresponding number of abnormalities involved in each category.

[0190] The abnormal records during the conversion process are categorized and organized into three types: field mapping abnormalities, data type conversion abnormalities, and relationship adjustment abnormalities. The metamodel template identifiers involved in each type of abnormality are statistically analyzed. For example, in field mapping abnormalities, template C is involved 5 times, template D is involved 3 times, and so on, along with the corresponding number of abnormal occurrences.

[0191] Step S1562: For each type of anomaly, retrieve the data transformation rule information and data structure description information of the metamodel template, and compare the anomaly phenomenon in the anomaly record with the rules and standards of the metamodel template;

[0192] For each type of anomaly, the metamodel template is used to retrieve the data transformation rules and data structure description information of that template. For example, for template C involved in field mapping anomalies, its field mapping relationships, type conversion requirements, association standards, and other information are retrieved. Then, the anomalies in the anomaly records are compared with the rules and standards of template C.

[0193] If the field mapping anomaly manifests as the original field not being able to find a corresponding standard field, and the comparison reveals that the template field mapping relationship does not contain the mapping entry corresponding to the original field, then the anomaly is determined to be due to the incomplete conversion rules of the template, and optimization suggestions for the conversion rules to supplement the original field mapping entry are generated.

[0194] If a field mapping anomaly manifests as the original field failing to find a corresponding standard field, and a comparison reveals that the template's field mapping relationship lacks a corresponding mapping entry for that original field, then the anomaly is determined to stem from an incomplete conversion rule in the template. For example, if an original field "Order Status" lacks a corresponding standard field in the field mapping relationship of template C, then an optimization suggestion for the conversion rule should be generated to supplement the original field's mapping entry. This suggestion would be to add the standard field "OrderStatus" corresponding to "Order Status" to the field mapping relationship of template C.

[0195] If the data type conversion anomaly manifests as the original data type exceeding the template type constraint range, and the comparison reveals that the template data type constraint range does not cover the original data type, then the anomaly is determined to originate from a mismatch in the template's structural standards, and a structural standard adjustment suggestion to expand the template data type constraint range is generated.

[0196] If a data type conversion anomaly manifests as the original data type exceeding the template's type constraints, and a comparison reveals that the template's data type constraints do not cover the original data type, then the anomaly is determined to stem from a mismatch in the template's structural standards. For example, if an original data type is "decimal," but template C's data type constraints do not include this type, then a structural standard adjustment suggestion is generated to expand the data type constraints of template C, recommending the addition of the "decimal" type to template C's data type constraints.

[0197] If the abnormal behavior of the relationship adjustment is that the original field relationship method cannot be adapted to the template relationship standard, and the comparison finds that the template relationship standard does not include the adaptation rules of the target class relationship method, then the abnormality is determined to be due to the incomplete conversion rules of the template, and the conversion rule optimization suggestion of adding the target class relationship method adaptation rule is generated.

[0198] If the anomaly in the relationship adjustment manifests as the original field association method failing to adapt to the template's association standard, and a comparison reveals that the template's association standard lacks an adaptation rule for the target class's association method, then the anomaly is determined to stem from incomplete conversion rules in the template. For example, if the original field association method is a special type of many-to-many association, and template C's association standard lacks an adaptation rule for this type of association, then an optimization suggestion should be generated to add an adaptation rule for this type of many-to-many association in template C's association standard.

[0199] Step S1563: For each generated optimization or adjustment suggestion, mark the corresponding anomaly record number, the relevant meta-model template identifier, and the specific steps for implementing the suggestion;

[0200] For each generated optimization or adjustment suggestion, mark the corresponding exception record number, such as exception record 001, 002, etc.; mark the relevant metamodel template identifier, such as template C, template D, etc.; mark the specific steps for implementing the suggestion, such as the steps for supplementing field mapping entries are to open the field mapping relationship configuration file of template C, add the corresponding mapping entries, and then save and re-verify.

[0201] Step S157: Integrate template conversion rule optimization suggestions and template structure standard adjustment suggestions to form a metamodel library optimization scheme. The metamodel library optimization scheme includes the metamodel template identifier to be optimized, specific optimization content, and optimization implementation steps.

[0202] All template conversion rule optimization suggestions and template structure standard adjustment suggestions are integrated to form a metamodel library optimization plan. The plan includes the identifiers of the metamodel templates to be optimized, such as template C and template D; specific optimization content, such as supplementing field mapping entries and expanding the range of data type constraints; and optimization implementation steps, such as opening the template configuration file, making corresponding adjustments, saving, and verifying. Simultaneously, the priority of each optimization suggestion needs to be evaluated within the metamodel library optimization plan, for example, based on factors such as the frequency of anomalies and the degree of impact on data standardization, determining which optimization suggestions should be implemented first. For the implementation steps, the operation objects, operation methods, and operation tools for each step need to be described in detail to ensure that those skilled in the art can optimize the metamodel library according to the plan. For example, for the optimization suggestion of supplementing field mapping entries, the operation object is the field mapping relationship configuration file of the metamodel template; the operation method is to open the file with a text editor and add the mapping relationship entries between the original fields and standard fields at the specified locations; the operation tool can be common text editing software or a metamodel library management tool. After integrating all optimization suggestions and providing detailed instructions on implementation steps, the metamodel library optimization scheme is formed. This scheme will guide subsequent updates and improvements to the metamodel library to enhance the efficiency and quality of standardized processing of multi-source heterogeneous data.

[0203] Please see Figure 2 The schematic block diagram of the multi-source heterogeneous data standardization processing terminal 100 based on metamodel adaptive matching provided in this application embodiment for executing the above-described multi-source heterogeneous data standardization processing method based on metamodel adaptive matching may include a communication unit 110, a machine-readable storage medium 120, and a processor 130.

[0204] In this embodiment, both the machine-readable storage medium 120 and the processor 130 are located in the multi-source heterogeneous data standardization processing terminal 100 based on metamodel adaptive matching and are separately configured. Alternatively, the machine-readable storage medium 120 can also be integrated into the processor 130 and can communicate and interact with external systems through the communication unit 110. The machine-readable storage medium 120 is used to store machine-executable instructions for executing the scheme of this application, and the processor 130 is used to execute the machine-executable instructions stored in the machine-readable storage medium 120 to implement the multi-source heterogeneous data standardization processing method based on metamodel adaptive matching provided in the aforementioned method embodiments.

[0205] This invention has many specific applications. The above description is only a preferred embodiment of this invention. It should be noted that for those skilled in the art, several improvements can be made without departing from the principle of this invention, and these improvements should also be considered within the scope of protection of this invention.

Claims

1. A method for standardizing multi-source heterogeneous data based on meta-model adaptive matching, characterized in that, The steps are as follows: Acquire a multi-source heterogeneous data set, which contains raw data units from different data sources and with different data structures, and record the data source attributes and collection scenario attributes of each raw data unit; Based on the data source attributes and collection scenario attributes of the multi-source heterogeneous data set, the pre-adapted metamodel library is adjusted to generate a pre-adapted metamodel library. The pre-adapted metamodel library contains initial metamodel templates designed for different data structure types. Each initial metamodel template contains data structure description information, data type constraint information, and data conversion rule information. Data structure features are extracted from each original data unit in the multi-source heterogeneous data set to generate multi-dimensional structural features for each original data unit. Based on the multi-dimensional structural features, multi-round adaptive matching is performed with the meta-model templates in the pre-adapted meta-model library to generate meta-model matching results and matching optimization suggestions for each original data unit. Based on the metamodel matching result, the data transformation rule information of the corresponding metamodel template is called, and the matching optimization suggestion is combined to perform dynamic standardization transformation on each original data unit, generating a standardized data unit and a transformation process record corresponding to each original data unit. Collect standardized data units and transformation process records corresponding to all original data units, construct a standardized data set, perform correlation analysis on the standardized data set and transformation process records, and generate a report on the standardized processing results of multi-source heterogeneous data and an optimization scheme for the meta-model library.

2. The method for standardizing multi-source heterogeneous data based on meta-model adaptive matching according to claim 1, characterized in that, The step of performing meta-model pre-adaptation adjustment on the preset meta-model library based on the data source attributes and collection scenario attributes of the multi-source heterogeneous data set to generate a pre-adapted meta-model library includes: Extract the data source attributes of all original data units in the multi-source heterogeneous data set, classify and statistically analyze the number of original data units and the types of data structure differences corresponding to different data sources, and generate a data source structure distribution table. Extract the collection scenario attributes of all original data units in the multi-source heterogeneous data set, analyze the variation law of the number of fields and the characteristics of field association under different collection scenarios, and generate a scenario field feature table; Retrieve all initial metamodel templates in the preset metamodel library and extract the range of the number of fields and the type of field association from the data structure description information of each initial metamodel template; The data structure difference type in the data source structure distribution table is compared with the field association type of each initial metamodel template, and the initial metamodel templates with mismatched field association types are marked. The variation pattern of the number of fields in the scene field feature table is compared with the range of the number of fields in each initial metamodel template, and initial metamodel templates with mismatched ranges of the number of fields are marked. For initial metamodel templates where the field association methods do not match, adjust the field association method types in their data structure description information so that the adjusted field association method types are adapted to the structural difference types of the corresponding data sources; For initial metamodel templates where the number of fields in the tags does not match, adjust the number of fields in their data structure description information to make the adjusted number of fields adapt to the field number variation pattern of the corresponding collection scenario. After the adjustments are completed, the adaptability of each adjusted metamodel template is verified by selecting the original data units from the corresponding data sources and collection scenarios, and testing the template’s coverage of the data structure. Based on the adaptability verification results, fine-tune the field constraints and conversion rule priorities of the metamodel template until the template's coverage of the data structure in the corresponding data source and collection scenario reaches the preset deviation range. Integrate all adjusted and adapted metamodel templates to build a pre-adapted metamodel library, and record the adjustment content of each metamodel template as well as the data source and collection scenario information for adaptation.

3. The method for standardizing multi-source heterogeneous data based on meta-model adaptive matching according to claim 2, characterized in that, The step of fine-tuning the field constraints and transformation rule priorities of the metamodel template based on the adaptability verification results until the template's coverage of the data structure in the corresponding data source and acquisition scenario reaches a preset deviation range includes: Extract the coverage of each metamodel template to the original data units in the corresponding data source and collection scenario during the adaptability verification process, and count the number of original data units not covered by the template and the reasons for not being covered. If the reason for the lack of coverage is that the field constraints of the template are too strict, then analyze the field attributes of the original data units that are not covered, adjust the data type range or format requirements in the template field constraints, and expand the range of compatibility of the constraints. If the reason for the lack of coverage is that the priority setting of the template's transformation rules is improper, causing some original data units to fail to be transformed according to the rules, then the applicable scenarios of the transformation rules should be re-evaluated, and the priority order of different transformation rules should be adjusted so that rules with a wide range of applicable scenarios are given priority. After the adjustment is completed, the original data units that were not covered are selected for adaptation testing again. The number of original data units covered by the template after adjustment is counted, and the degree of coverage is calculated. If the coverage does not reach the preset deviation range, repeat the above steps of adjusting field constraints or adjusting conversion rule priority until the template's coverage of the original data units in the corresponding data source and collection scenario reaches the preset deviation range. Record the specific details of each adjustment, the changes in coverage after the adjustment, and the information of the original data units used for testing, forming a template fine-tuning record.

4. The method for standardizing multi-source heterogeneous data based on meta-model adaptive matching according to claim 1, characterized in that, The process involves extracting data structure features from each original data unit in the multi-source heterogeneous data set to generate multi-dimensional structural features for each original data unit. Based on these multi-dimensional structural features, multiple rounds of adaptive matching are performed with meta-model templates in the pre-adapted meta-model library to generate meta-model matching results and matching optimization suggestions for each original data unit. This includes: Select a single original data unit from the multi-source heterogeneous data set, perform field hierarchical parsing on the original data unit, determine the hierarchical affiliation and depth of each field, and generate a field hierarchical structure table; Analyze the reference and dependency relationships between different fields in the original data unit, count the number of field associations and the length of the association path, and generate a field association table; Extract the data type and data format of each field in the original data unit, count the proportion of different data types in all fields, and generate a data type distribution table; Based on the field hierarchy structure table, field association table, and data type distribution table, a multi-dimensional structural feature of the original data unit is constructed. The multi-dimensional structural feature includes hierarchical features, association features, and type features. Retrieve all metamodel templates from the pre-adapted metamodel library, and extract the hierarchical standards, association standards, and type standards corresponding to the data structure description information of each metamodel template; In the first round of matching, the hierarchical features of the multi-dimensional structural features are compared with the hierarchical standards of each meta-model template. The hierarchical fit is generated by comparing the number of corresponding field hierarchies with the degree of matching of hierarchical depth. A set of candidate templates whose hierarchical fit meets the first preset threshold is then selected. In the second round of matching, the association features of the multi-dimensional structural features are compared with the association criteria of each template in the candidate template set. The association fit is generated by comparing the matching degree of the number of field associations and the fit of the association path length. The secondary candidate template set that meets the second preset threshold is then selected. In the third round of matching, the type features of the multi-dimensional structural features are compared with the type standards of each template in the secondary candidate template set. The type fit is generated by comparing the data type proportion matching degree and the data format conformity degree. The template with the highest type fit is selected as the initial matching template. Analyze the adaptation deviation between the preliminary matching template and the multi-dimensional structural features. If the deviation is within the preset deviation range, the preliminary matching template is determined as the final matching template, and a meta-model matching result containing the final matching template identifier and the adaptation degree of each round is generated. If the deviation exceeds the preset deviation range, matching optimization suggestions are generated based on the deviation content. The matching optimization suggestions include the template standard items that need to be adjusted and the adjustment direction. The preliminary matching template and the matching optimization suggestions are used together as the meta-model matching result.

5. The method for standardizing multi-source heterogeneous data based on meta-model adaptive matching according to claim 4, characterized in that, The step of constructing the multi-dimensional structural features of the original data unit based on the field hierarchy structure table, field association table, and data type distribution table includes: Extract the hierarchy depth of each field from the field hierarchy structure table, calculate the average and maximum values ​​of the hierarchy depth of all fields, and use the average hierarchy depth, the maximum hierarchy depth, and the distribution of the number of fields at different hierarchy depths as the constituent elements of the hierarchy feature. Extract the number of field associations and the length of the association path from the field association table, calculate the average number of associations for each field and the median of all association path lengths, and use the average number of field associations, the median of the association path length, and the percentage of fields with more than a preset number of associations as the constituent elements of the association feature. Extract the field proportions of different data types from the data type distribution table, filter data types whose field proportions exceed a preset proportion threshold to form a list of main data types, and use the list of main data types, the proportion values ​​of each data type in the list of main data types, and the number of data types as the constituent elements of the type features; The constituent elements of hierarchical features, relational features, and type features are processed in a unified format, and each element is described using the same data representation form. The integrated hierarchical features, association features, and type features are combined to form the multi-dimensional structural features of the original data unit, and a unique identifier is assigned to each dimension feature.

6. The method for standardizing multi-source heterogeneous data based on meta-model adaptive matching according to claim 1, characterized in that, The step involves calling the data transformation rule information of the corresponding metamodel template based on the metamodel matching result, and combining the matching optimization suggestions to dynamically standardize each original data unit, generating a standardized data unit and a transformation process record corresponding to each original data unit, including: Extract the final matching template from the meta-model matching results, and retrieve the data transformation rule information contained in the final matching template. The data transformation rule information includes the transformation operation sequence, field mapping relationship and content adjustment method. If the metamodel matching result contains matching optimization suggestions, then adjust the sequence order of transformation operations or the priority of field mapping relationships in the data transformation rule information according to the matching optimization suggestions to generate optimized data transformation rule information; According to the transformation operation sequence in the optimized data transformation rule information, the fields of the original data unit are mapped, the original field names are replaced with standard field names according to the field mapping relationship, and the field mapping correspondence is recorded; After completing the field mapping process, the data type of each field is converted. According to the type conversion requirements in the data conversion rule information, the original data type is converted into the standard data type. If a data format mismatch occurs during the conversion process, the data format is corrected according to the content adjustment method, and the details of data type conversion and format correction are recorded. After the data type conversion is completed, the field association relationship of the original data unit is adjusted. According to the association relationship standard in the data conversion rule information, the reference relationship and dependency relationship between fields are adjusted to make the field association conform to the association relationship standard, and the association relationship adjustment steps are recorded. After completing all transformation operations, according to the data structure description information of the final matching template, all processed fields are reorganized to form standardized data units, and the field composition and data content of the standardized data units are recorded. Integrate field mapping correspondences, data type conversion and format correction details, association adjustment steps, and standardized data unit composition information to generate a conversion process record for the original data unit.

7. The method for standardizing multi-source heterogeneous data based on meta-model adaptive matching according to claim 6, characterized in that, If the metamodel matching result contains matching optimization suggestions, the order of transformation operation sequences or the priority of field mapping relationships in the data transformation rule information are adjusted according to the matching optimization suggestions to generate optimized data transformation rule information, including: Analyze the matching optimization suggestions, determine the template standard items that need to be adjusted in the matching optimization suggestions, and determine whether the components of the transformation rule corresponding to the template standard item are a transformation operation sequence or a field mapping relationship; If the adjustment needs to be made to the sequence of transformation operations, then analyze the adjustment direction in the matching optimization suggestions. If the adjustment direction is to increase the execution priority of the target class transformation operation, then move the target class transformation operation to the beginning of the transformation operation sequence. If the adjustment direction is to split a transformation operation, then split the transformation operation into multiple sub-operations and insert them into the transformation operation sequence in logical order. If the priority of field mapping relationships needs to be adjusted, the adjustment basis in the matching optimization suggestions is analyzed. If the adjustment basis is the semantic relevance of fields, the preset semantic relevance rule library is called to score the fields, and the field mapping relationships are sorted in descending order according to the score results to determine the execution order. If the adjustment basis is the proportion of field data volume, the field mapping relationships are sorted in descending order directly according to the value of the proportion of each field data volume to determine the execution order. After the adjustment is completed, the optimized conversion operation sequence and field mapping relationship are simulated. Some fields of the original data unit are selected for trial conversion, and it is observed whether any new adaptation deviations occur during the trial conversion process. If no new adaptation deviations occur, the optimized data conversion rule information is confirmed to be valid; if new adaptation deviations occur, the conversion operation sequence or field mapping relationship is fine-tuned according to the deviation situation, and the conversion is performed again until no new adaptation deviations occur. Record the differences in data conversion rule information before and after adjustment, the deviations during the trial conversion process, and the final optimization results to form a conversion rule optimization record, which serves as a component of the conversion process record.

8. The method for standardizing multi-source heterogeneous data based on meta-model adaptive matching according to claim 1, characterized in that, The process involves collecting standardized data units and transformation process records corresponding to all original data units, constructing a standardized data set, performing correlation analysis on the standardized data set and transformation process records, and generating a report on the standardization processing results of multi-source heterogeneous data and an optimization scheme for the meta-model library, including: Traverse all the standardized data units and transformation process records corresponding to the original data units, collect the complete data content of each standardized data unit and the corresponding transformation process record, and establish a one-to-one correspondence between standardized data units and transformation process records; All standardized data units are grouped according to the data source attributes of their corresponding original data units, forming standardized data groups divided by data source. The number of standardized data units and the conversion completion rate corresponding to each data source are counted. Perform field consistency analysis on the standardized data units in each standardized data group, compare the data content format of the same standard fields in different standardized data units under the same data source, and count the proportion of fields with uniform format; Analyze all conversion process records, extract records of field mapping anomalies, data type conversion failures, and difficulties in adjusting relationships that occurred during the conversion process, and count the number of occurrences of each type of anomaly and the corresponding data source attributes and metamodel template types; Based on the above statistics of the number of standardized data units, conversion completion rate, proportion of fields with unified format, and occurrence of various anomalies, a multi-source heterogeneous data standardization processing result report is generated. The processing result report includes the standardization processing effectiveness of each data source, the main anomaly types in the conversion process, and the standardized data quality assessment. Based on the anomaly records and standardized data quality assessment results during the conversion process, the metamodel template problem causing the anomaly is located. If the anomaly originates from incomplete template conversion rules, optimization suggestions for template conversion rules are generated; if the anomaly originates from mismatched template structural standards, suggestions for adjusting template structural standards are generated. The optimization suggestions for template conversion rules and the adjustment suggestions for template structure standards are integrated to form an optimization scheme for the metamodel library. The optimization scheme for the metamodel library includes the metamodel template identifiers to be optimized, specific optimization content, and optimization implementation steps.

9. The method for standardizing multi-source heterogeneous data based on meta-model adaptive matching according to claim 8, characterized in that, Based on the abnormal records and standardized data quality assessment results during the conversion process, the metamodel template problem causing the abnormality is located. If the abnormality originates from the imperfect conversion rules of the template, optimization suggestions for the template conversion rules are generated. If the anomaly stems from a mismatch in the template's structural standards, then template structural standard adjustment suggestions will be generated, including: The abnormal records during the transformation process are classified and organized into three categories: field mapping abnormalities, data type conversion abnormalities, and relationship adjustment abnormalities. The meta-model template identifiers and corresponding number of abnormalities involved in each category are counted. For each type of anomaly, retrieve the data transformation rule information and data structure description information of the metamodel template, and compare the anomaly phenomenon in the anomaly record with the rules and standards of the metamodel template. If the field mapping anomaly manifests as the original field not being able to find a corresponding standard field, and the comparison reveals that the template field mapping relationship does not contain the mapping entry corresponding to the original field, then the anomaly is determined to be due to the incomplete conversion rules of the template, and optimization suggestions for the conversion rules to supplement the original field mapping entry are generated. If the data type conversion anomaly manifests as the original data type exceeding the template type constraint range, and the comparison reveals that the template data type constraint range does not cover the original data type, then the anomaly is determined to originate from a mismatch in the template's structural standards, and a structural standard adjustment suggestion to expand the template data type constraint range is generated. If the abnormal behavior of the relationship adjustment is that the original field relationship method cannot be adapted to the template relationship standard, and the comparison finds that the template relationship standard does not include the adaptation rules of the target class relationship method, then the abnormality is determined to be due to the incomplete conversion rules of the template, and the conversion rule optimization suggestion of adding the target class relationship method adaptation rule is generated. For each generated optimization or adjustment suggestion, mark the corresponding anomaly record number, the relevant meta-model template identifier, and the specific steps for implementing the suggestion.

10. A multi-source heterogeneous data standardization processing terminal based on meta-model adaptive matching, characterized in that, include: One or more processors; A machine-readable storage medium for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the multi-source heterogeneous data standardization processing method based on meta-model adaptive matching as described in any one of claims 1-9.

Images