Standard adaptive execution method and equipment based on data standard management

Abstract

The invention provides a standard adaptive execution method and equipment based on data standard management, and belongs to the technical field of data management. The method comprises the following steps: in response to a standardization request for a service system from a user, generating a corresponding mapping relation library based on a preset root library and a source system field; wherein the mapping relation library at least comprises a mapping relation, established on the basis of a preset semantic matching model, between the source system field and the standard field. And converting each source data value of the business system according to the mapping relation library, and executing real-time verification based on a preset multi-level value domain enumeration constraint before the source data values are written into the target library, so as to intercept conversion abnormal data according to a verification result and generate an abnormal snapshot. And when it is determined that the data standard attribute change event exists or the abnormal snapshots meeting the preset conditions exist, determining a corresponding update strategy type based on the change content of the data standard version change or the root cause classification result of each abnormal snapshot, so as to perform standardized execution according to a corresponding update strategy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data management, and in particular to a standard adaptive execution method and device based on data standard management. BACKGROUND

[0002] With the continuous development of digitization, in the current data governance field, especially in data-intensive enterprises (such as finance, medical treatment, etc.), there is a common challenge of needing to unify scattered business system data to enterprise-level data standards.

[0003] At present, the existing scheme mainly relies on manual configuration of field mapping relationship, and completes source data conversion and writing standard library landing mark execution work through ETL tools. This way needs to consume a large amount of professional human resources, and due to the cognitive differences between business personnel and technical personnel, the mapping error rate is high, and the mapping configuration efficiency is low.

[0004] Moreover, when the enterprise data standard version changes, the existing mapping link needs to be completely restructured. At present, the way of batch compliance verification through SQL scripts also leads to insufficient timely landing quality monitoring, affecting user system use experience. SUMMARY

[0005] The embodiments of the present application provide a standard adaptive execution method and device based on data standard management, which is used to solve the technical problems that the current data standard management relies too much on manual landing and mapping, is low in efficiency, cannot cope with the difficulties brought by standard changes, and landing quality monitoring is not timely, which brings poor user experience.

[0006] In one aspect, the embodiments of the present application provide a standard adaptive execution method based on data standard management, which is applied to a data standard management system; the method comprises:

[0007] In response to a standardization request from a user to a business system, a corresponding mapping relationship library is generated based on a preset root library and source system fields; wherein the mapping relationship library at least includes the mapping relationship between the source system fields and the standard fields established based on a preset semantic matching model;

[0008] According to the mapping relationship library, the source data values of the business system are converted, and before writing to the target library, real-time verification is performed based on a preset multi-level value domain enumeration constraint to intercept conversion abnormal data and generate an abnormal snapshot according to the verification result; the preset multi-level value domain enumeration constraint at least corresponds to the following value domains: basic value domain, business value domain and standard value domain;

[0009] When it is determined that there is a data standard attribute change event or there is an abnormal snapshot that meets a preset condition, a corresponding update strategy type is determined based on a change content of the data standard version change or a root cause classification result of each abnormal snapshot, so that the standardization is performed according to the corresponding update strategy; wherein the update strategy type at least includes: field value conversion script, mapping rule update instruction.

[0010] In an implementation manner of the present application, a corresponding mapping relationship library is generated based on a preset root library and a source system field, specifically including:

[0011] The source system field is parsed according to the preset root library to split the source system field into a root sequence, and the preset semantic matching model is input to construct a corresponding root semantic vector; wherein the root semantic vector is obtained by splicing an entity mapping sub-vector and a relationship mapping sub-vector;

[0012] The vector similarity of the root semantic vector and a preset standard field feature vector is calculated;

[0013] When the vector similarity is greater than a preset similarity threshold, a mapping relationship between the source system field and the corresponding standard field is established to generate the mapping relationship library.

[0014] In an implementation manner of the present application, real-time verification is performed based on a preset multi-level value domain enumeration constraint to intercept conversion abnormal data and generate an abnormal snapshot according to the verification result, specifically including:

[0015] The source data value is verified at a level according to the preset multi-level value domain enumeration constraint;

[0016] When the source data value does not pass at any verification level, the writing of the source data value to the target library is terminated, and the abnormal snapshot is generated.

[0017] In an implementation manner of the present application, the source data value is verified at a level, specifically including:

[0018] First level: determining whether the source data value meets a basic value domain constraint, the basic value domain constraint including a data type constraint, a length constraint and a format constraint;

[0019] Second level: if the first level verification is passed, determining whether the source data value is in a first enumeration value range allowed by the business value domain constraint;

[0020] Third level: if the second level verification is passed, determining whether the source data value is in a second enumeration value range allowed by the standard value domain constraint.

[0021] In an implementation form of the application, the abnormal snapshot comprises at least one or more of the following information: an abnormal data value and its original metadata, a specific value range constraint condition violated, an associated mapping relationship label, and an abnormality check level identifier.

[0022] In an implementation form of the application, the method further comprises:

[0023] When it is determined that the source data value does not meet the type matching condition corresponding to the data type constraint, a preset conversion rule is executed to convert a source data type corresponding to the source data value into a target field type; the preset conversion rule comprises at least one or more of the following: string to numerical type conversion, numerical type to string conversion, date format standardization conversion, and Boolean type to integer type mapping conversion.

[0024] In an implementation form of the application, before determining that the data standard attribute change event exists, the method further comprises:

[0025] monitoring a standard field template and determining whether an attribute of the standard field template is modified;

[0026] if yes, marking a standard set associated with the standard field model and generating the data standard attribute change event.

[0027] In an implementation form of the application, the determination that the abnormal snapshot meeting the preset condition exists comprises:

[0028] accumulating a number of abnormal snapshots of the abnormal snapshots and determining a business standard type corresponding to each of the abnormal snapshots;

[0029] in a case where the number of abnormal snapshots is greater than a predetermined number of values and / or there is the abnormal snapshot of the selected business standard type, it is determined that the preset condition is met.

[0030] In an implementation form of the application, based on the change content of the data standard version change or the root cause classification result of each of the abnormal snapshots, a corresponding update strategy type is determined to perform standardization according to a corresponding update strategy, specifically comprising:

[0031] the update strategy type is determined according to the change content or the root cause classification result through a strategy determination rule; wherein the strategy determination rule comprises: when the change content involves a value range or a data structure, determining that the update strategy type is a field value conversion script, when the change content only involves a semantic rule or an association, determining that the update strategy type is a mapping rule update instruction, when the root cause classification result is a standard definition defect, determining that the update strategy type is a field value conversion script, and when the root cause classification result is a mapping rule error, determining that the update strategy type is a mapping rule update instruction.

[0032] In a case where the update strategy type is a field value conversion script, a field value conversion script is generated and executed to convert the source data value;

[0033] In a case where the update strategy type is the mapping rule update instruction, the mapping rule update instruction is generated and the parameters of the preset semantic matching model are adjusted.

[0034] In another aspect, the embodiments of the present application also provide a standard adaptive execution device based on data standard management, which is applied to a data standard management system; the device comprises:

[0035] at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the standard adaptive execution method based on data standard management as described above.

[0036] Compared with the prior art, the present application has the following remarkable effects:

[0037] Through the above technical solution, the mapping relationship library is automatically generated by the preset semantic matching model, replacing manual configuration of field mapping, and the multi-level value domain verification mechanism is used to realize abnormal interception, thereby significantly improving the data standardization efficiency and accuracy. At the same time, the update strategy is determined based on the data standard attribute change event or abnormal snapshot root cause, and the adaptive governance of the abnormality is realized. Thus, the technical problems that the current data standard management excessively relies on manual marking and mapping, has low efficiency, cannot cope with the difficulties brought by standard changes, and cannot timely monitor the quality of marking, thereby bringing poor user experience are solved. BRIEF DESCRIPTION OF DRAWINGS

[0038] The accompanying drawings, which are included to provide a further understanding of the present application, constitute a part of the present application and help to explain the present application and its specification, and do not limit the present application in any way. In the drawings:

[0039] Figure 1 is a flowchart of the standard adaptive execution method based on data standard management in the embodiments of the present application;

[0040] Figure 2 is a structural diagram of the standard adaptive execution device based on data standard management in the embodiments of the present application. DETAILED DESCRIPTION

[0041] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in combination with specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0042] The embodiments of the present application provide a standard adaptive execution method and device based on data standard management, to solve the technical problems that the current data standard management excessively relies on manual work for falling and mapping, is inefficient, cannot cope with the difficulties brought by standard changes, and the falling quality monitoring is not timely, which brings poor user experience.

[0043] The various embodiments of the present application will be described in detail below in combination with the drawings.

[0044] The embodiments of the present application provide a standard adaptive execution method based on data standard management, as shown in Figure 1 The method is applied to a data standard management system, and the method can include steps S101-S103:

[0045] S101, the server generates a corresponding mapping relationship library based on a preset root library and a source system field in response to a standardization request of a user to a business system.

[0046] The mapping relationship library at least includes a mapping relationship between the source system field and the standard field established based on a preset semantic matching model.

[0047] It should be noted that the server is an execution subject of the standard adaptive execution method based on data standard management, which is only an example, and the execution subject is not limited to the server, and the present application does not make a specific limitation.

[0048] The data standard management system at least includes: a code table management module for managing multi-level code table directories and code values, supporting multi-level code table directory management, and users can create a maximum of 5-level code table directories, and a maximum of 10000 code values can be added under each directory; a root management module for managing business nouns, physical tables, field specification naming and standardized translation; a standard template management module for defining attribute information to be filled in when creating a data standard; a standard set management module for managing a group of standards similar or consistent with business meaning; a data standard management module for standardizing definition and application of data standards; and a falling standard evaluation management module for defining matching relationships between standard attributes and asset object metadata fields and evaluating mapping relationships.

[0049] The user can establish a communication connection with the server through a mobile phone, a computer or the like, and send a standardized request based on the business system standardization to the server. For example, data of a certain medical system is standardized, and data of a certain financial system is standardized.

[0050] In the embodiment of the present application, a corresponding mapping relationship library is generated based on the preset root library and the source system field, specifically including:

[0051] The source system field is parsed according to the preset root library to split the source system field into a root sequence, and a corresponding root semantic vector is constructed by inputting the preset semantic matching model. The root semantic vector is obtained by splicing an entity mapping sub-vector and a relationship mapping sub-vector. The vector similarity of the root semantic vector and the preset standard field feature vector is calculated. In the case where the vector similarity is greater than a preset similarity threshold, the mapping relationship between the source system field and the corresponding standard field is established to generate the mapping relationship library.

[0052] In other words, the preset root library can be used to parse the source system field, which can be understood as splitting the source system field into one or more roots consistent with the roots existing in the preset root library and assembling them into a root sequence. The preset root library can be obtained through the above root management module.

[0053] Subsequently, the preset semantic matching model is used to construct an entity mapping sub-vector based on an entity mapping rule and a relationship mapping sub-vector based on a relationship mapping rule. According to a preset splicing weight, the entity mapping sub-vector and the relationship mapping sub-vector are weighted and spliced to obtain a root semantic vector.

[0054] The preset semantic matching model can be a pre-trained deep learning model or other models, which are not limited in the present application. The entity mapping rule assigns a preset entity weight to each root in the field; the relationship mapping rule calculates the correlation strength of the root, and constitutes a relationship mapping sub-vector containing the correlation strength, which can be obtained according to the product value of the root co-occurrence frequency and the preset business weight. The preset entity weight, the preset business weight and the preset splicing weight can be obtained by expert experience, which are not limited in the present application. The root semantic vector containing entity mapping and relationship mapping can effectively improve the matching accuracy of complex fields.

[0055] Subsequently, the server will perform vector similarity calculation through a preset semantic matching model, which can be calculated by a cosine similarity formula or other similarity calculation formulas, which are not specifically limited herein. When the vector similarity is greater than a preset similarity threshold, the server will screen the standard field corresponding to the vector similarity as the source system field, and establish a mapping relationship, thereby constructing a mapping relationship library containing the mapping relationship. That is, the semantic matching model uses entity mapping rules and relationship mapping rules to construct the field feature vector (root semantic vector) corresponding to the source system field, and calculates the vector similarity to determine the mapping matching degree.

[0056] For example, the source system field: medical patient, clinical number, social insurance, the corresponding root sequence is [patient, number, social security], after executing the above mapping, the standard field thereof is obtained as patient-social security, and finally the mapping relationship of (the number of patient A is B, and patient A has social security)-(patient-social security) is obtained.

[0057] For the source system field that does not meet the preset similarity threshold, the server can generate a prompt information and send it to the user terminal, so as to manually intervene in the mapping relationship construction. The above-mentioned preset similarity threshold can be set by the user according to the actual use scene, which is not specifically limited herein.

[0058] S102, the server converts each source data value of the business system according to the mapping relationship library, and performs real-time verification based on a preset multi-level value domain enumeration constraint before writing into the target library, so as to intercept and generate an exception snapshot according to the verification result.

[0059] The preset multi-level value domain enumeration constraint at least corresponds to the following value domains: basic value domain, business value domain and standard value domain.

[0060] After generating the mapping relationship library, each source data value in the business system can be converted into standard data, but there are cases such as error mapping and invalid mapping during the mapping conversion process, so verification work is needed. In the embodiment of the present application, real-time verification is performed based on a preset multi-level value domain enumeration constraint, so as to intercept and generate an exception snapshot according to the verification result, which specifically includes:

[0061] According to the preset multi-level value domain enumeration constraint, the source data value is checked at different levels. When the source data value does not pass at any check level, the writing of the source data value into the target library is terminated, and an exception snapshot is generated.

[0062] That is, the preset multi-level value domain enumeration constraint is adopted to perform multi-level verification on the converted source data value, and to intercept abnormal data.

[0063] Among them, the hierarchical verification of the source data value specifically includes:

[0064] The first level: determine whether the source data value meets the basic value domain constraint, which includes data type constraint, length constraint and format constraint. That is, verify whether the data type, length and format of the source data value meet the rules. The data type can include integer, character and date; the length constraint is in units of bytes; and the format constraint can be defined by a regular expression.

[0065] The second level: if the first level check passes, determine whether the source data value is within the first enumeration value range allowed by the business value domain constraint. That is, if the basic value domain check passes, verify whether the data value is within the first enumeration value range allowed by the source system business rules. The first enumeration value range can be obtained by dynamically interfacing the source system metadata database and updated in real time, or can be defined by users based on expert experience, which is not limited here.

[0066] The third level: if the second level check passes, determine whether the source data value is within the second enumeration value range allowed by the standard value domain constraint. That is, if the business value domain check passes, verify whether the data value meets the second enumeration value range constraint defined by the industry standard or enterprise standard. The second enumeration value range can be obtained by users through the standard Application Programming Interface (API) to synchronize the external industry standard library, which is not limited here.

[0067] When the data value does not pass any check level, immediately terminate writing to the target library and generate an exception snapshot. The exception snapshot includes at least one or more of the following information: abnormal data value and its original metadata, specific value domain constraint condition violated, associated mapping relationship label (ID), and exception check level identifier.

[0068] For example, the exception snapshot = {

[0069] "Abnormal data value": original numerical value, / / such as ID card number '11010119990101ABCD'

[0070] "Specific value domain constraint condition violated": "basic value domain / length constraint (18 bits required)",

[0071] "Mapping relationship ID": "MAP_2024_ID_CARD", / / associated to a specific mapping rule

[0072] "Exception level": "basic value domain" / / identify the failure link

[0073] }.

[0074] The mapping relationship ID is associated with the mapping relationship in the mapping relationship library.

[0075] The application adopts multi-level value range sequential verification, not only verifies a single value range, improves the verification accuracy, and performs verification before writing into the target library, avoids pollution of the target library after writing, and affects the subsequent data standard execution. At the same time, hierarchical filtering can effectively reduce invalid calculation amount.

[0076] In addition, for individual conversion errors that do not meet the corresponding conditions of the multi-level value range enumeration constraint, a remediation rule can be set to avoid generating an abnormal snapshot to cause the standardized execution to wait. Therefore, the application provides the following embodiments, including:

[0077] When it is determined that the source data value does not meet the type matching condition corresponding to the data type constraint, a preset conversion rule is executed to convert the source data type corresponding to the source data value into the target field type. The preset conversion rule includes at least one or more of the following: string to numerical type conversion, numerical type to string conversion, date format standardization conversion, Boolean type to integer type mapping conversion.

[0078] That is, when the source data value is converted, if the source data value does not meet the type matching condition in the data type constraint, such as the type of the source data value and the target type being different in character type, integer type, floating point type, date type, Boolean type, etc., resulting in not meeting the category matching condition, the server can call the preset conversion rule to convert the source data value into the target field type, avoiding being intercepted as conversion exception data.

[0079] For example, when the source data value (such as the string `"123.45"`) does not match the target field data type (such as the floating point type `DECIMAL`), the operation protocol of automatic format (string to numerical type) conversion is executed according to the preset conversion rule.

[0080] Among them, the conversion of the string to the numerical type in the application is to remove the non-numeric characters and convert them to numerical values, such as removing the currency symbols ($￥€) and thousand separators (,), and converting negative number strings to negative numerical values (such as "(123)" to -123); numerical type to string conversion: zero padding according to the target length; date format standardization conversion: the source date string is parsed according to the preset template and converted to the target format; Boolean type to integer type mapping conversion: true value is mapped to 1, and false value is mapped to 0.

[0081] S103, the server determines the corresponding update strategy type based on the change content of the data standard version change or the root cause classification result of each abnormal snapshot when it is determined that there is a data standard attribute change event or there is an abnormal snapshot that meets the preset condition, to perform standardized execution according to the corresponding update strategy.

[0082] Among them, the update strategy type at least includes: field value conversion script, mapping rule update instruction.

[0083] In the embodiment of the present application, before determining that a data standard attribute change event occurs, the method further includes:

[0084] The standard field template is monitored and whether the attributes of the standard field template are modified is determined. If it is determined that the attributes of the standard field template are modified, the standard set associated with the standard field model is marked and a data standard attribute change event is generated.

[0085] Specifically, the data standard management module monitors standard field templates to determine if any attributes within the standard field template have changed. If so, the standard set associated with that attribute is marked and a data standard attribute change event is generated, allowing for timely policy updates to prevent conversion anomalies caused by subsequent template changes.

[0086] In the embodiment of the present application, determining whether there is an abnormal snapshot that meets the preset conditions specifically includes:

[0087] The number of abnormal snapshots of the abnormal snapshots is accumulated, and the business standard type corresponding to each abnormal snapshot is determined. If the number of abnormal snapshots is greater than a predetermined number value and / or there is an abnormal snapshot of the selected business standard type, it is determined that the preset condition is met.

[0088] In other words, the server will monitor the number of abnormal snapshots generated, and / or the business standard types corresponding to the abnormal snapshots, wherein the business standard types can be pre-set on the server by users or experts, and different abnormal snapshots can correspond to different business standard types, and users can select one or more business standard types as selected business standard types. When abnormal snapshots of these selected business standard types appear, it is determined that there are abnormal snapshots that meet the preset conditions. When the number of abnormal snapshots generated is greater than a predetermined number value, it will be determined that the preset conditions are met. In addition, the server can also combine the two conditions. When the number of abnormal snapshots generated is greater than a predetermined number value and abnormal snapshots of the selected business standard type exist, the preset conditions are met. The judgment rules of the specific preset conditions can be set by the user according to the actual usage scenario, and this application does not make specific restrictions on this.

[0089] In the embodiment of the present application, based on the content of the data standard version change or the root cause classification results of each abnormal snapshot, the corresponding update policy type is determined to perform standardized execution according to the corresponding update policy, specifically including:

[0090] According to the change content or the root cause classification result, an update strategy type is determined through a strategy determination rule. The strategy determination rule includes: when the change content involves a value range or a data structure, determining that the update strategy type is a field value conversion script; when the change content only involves a semantic rule or a correlation, determining that the update strategy type is a mapping rule update instruction; when the root cause classification result is a standard definition defect, determining that the update strategy type is a field value conversion script; and when the root cause classification result is a mapping rule error, determining that the update strategy type is a mapping rule update instruction. In a case where the update strategy type is the field value conversion script, a field value conversion script is generated and executed to convert a source data value. In a case where the update strategy type is the mapping rule update instruction, a mapping rule update instruction is generated and a parameter of a preset semantic matching model is adjusted.

[0091] In other words, the server can select a corresponding update strategy through the change content and the root cause classification result, so as to generate a field value conversion script or update a parameter of a preset semantic matching model.

[0092] The change type corresponding to the change content, such as value range change, can be determined by enumeration value increase or decrease or data type change; the data result change can be determined by field splitting / merging or adding a constraint; the semantic rule change can be determined by field renaming or synonym library updating; and the correlation change can be determined by adding / deleting a field relationship. The root cause classification result can be obtained through clustering, such as clustering analysis of an abnormal snapshot through a representative density-based clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, DBSCAN), to obtain a standard definition defect class: the same standard field is reported as an error in multiple system sets; and a mapping rule error class: a specific source system frequently violates a business value range.

[0093] Through the above technical solution, a mapping relationship library is automatically generated through a preset semantic matching model, replacing manual field mapping configuration, and an abnormality is intercepted through a multi-level value range verification mechanism, significantly improving data standardization efficiency and accuracy. At the same time, an update strategy is determined based on a data standard attribute change event or an abnormal snapshot root cause, to realize adaptive management of abnormalities. Thus, the technical problem that current data standard management relies too much on manual labeling and mapping, is low in efficiency, cannot cope with the difficulty brought by standard changes, and cannot timely monitor labeling quality, to bring poor user experience is solved.

[0094] Figure 2 A structure schematic diagram of a standard adaptive execution device based on data standard management provided by an embodiment of the present application is shown in Figure 2 The device is applied to a data standard management system. The device includes:

[0095] At least one processor; and a memory connected with the at least one processor in communication. Wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0096] In response to a standardized request from a user to a business system, a corresponding mapping relationship library is generated based on a preset root library and a source system field. The mapping relationship library at least includes a mapping relationship between the source system field and the standard field established based on a preset semantic matching model. According to the mapping relationship library, each source data value of the business system is converted, and before writing into the target library, real-time verification is performed based on a preset multi-level value domain enumeration constraint to intercept and generate an exception snapshot according to the conversion of the abnormal data and the verification result. The preset multi-level value domain enumeration constraint at least corresponds to the following value domains: basic value domain, business value domain and standard value domain. When it is determined that there is a data standard attribute change event or there is an abnormal snapshot that meets the preset condition, based on the change content of the data standard version change or the root cause classification result of each abnormal snapshot, a corresponding update strategy type is determined to perform standardization according to the corresponding update strategy. The update strategy type at least includes: field value conversion script, mapping rule update instruction.

[0097] Each of the embodiments in the present application is described in a progressive manner, and the same or similar parts of each embodiment can be referred to each other. Each embodiment focuses on the difference from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiment.

[0098] The device and the method provided by the embodiments of the present application are one-to-one correspondence, so the device also has the similar beneficial technical effects as the method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device will not be repeated here.

[0099] It should also be noted that the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or devices including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0100] The above merely provides an example of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the scope of claims of the present application.

Claims

1. A standard adaptive execution method based on data standard management, characterized in that: The method is applied to a data standard management system; the method comprises: In response to a user's request for standardization of a business system, a corresponding mapping relationship library is generated based on a preset root library and source system fields; wherein the mapping relationship library at least includes a mapping relationship between the source system fields and the standard fields established based on a preset semantic matching model; The source data values ​​of the business system are converted according to the mapping relationship library, and before being written into the target library, real-time verification is performed based on the preset multi-level value range enumeration constraints to intercept conversion abnormal data and generate abnormal snapshots based on the verification results; the preset multi-level value range enumeration constraints correspond to at least the following value ranges: basic value range, business value range and standard value range; When it is determined that there is a data standard attribute change event or an abnormal snapshot that meets the preset conditions, the corresponding update strategy type is determined based on the change content of the data standard version change or the root cause classification result of each abnormal snapshot, so as to perform standardized execution according to the corresponding update strategy; wherein, the update strategy type includes at least: field value conversion script, mapping rule update instruction.

2. A standard adaptive execution method based on data standard management according to claim 1, characterized in that: Generate a corresponding mapping relationship library based on the preset root library and source system fields, including: Parsing the source system field according to the preset root library to split the source system field into root sequences, and inputting the sequence into the preset semantic matching model to construct corresponding root semantic vectors; wherein the root semantic vectors are obtained by concatenating entity mapping subvectors and relationship mapping subvectors; Calculating the vector similarity between the root semantic vector and the preset standard field feature vector; In a case where the vector similarity is greater than a preset similarity threshold, a mapping relationship between the source system field and the corresponding standard field is established to generate the mapping relationship library.

3. The method for standard adaptive execution based on data standard management according to claim 1, characterized in that: Perform real-time validation based on preset multi-level value range enumeration constraints to intercept conversion anomaly data and generate anomaly snapshots based on the validation results, including: Performing hierarchical verification on the source data value according to the preset multi-level value range enumeration constraint; When the source data value fails any verification level, writing of the source data value into the target repository is terminated, and the exception snapshot is generated.

4. The method for standard adaptive execution based on data standard management according to claim 3, characterized in that: Performing hierarchical verification on the source data value, specifically including: First level: determining whether the source data value complies with basic value range constraints, which include data type constraints, length constraints, and format constraints; Second level: if the first level check is passed, determining whether the source data value is within the first enumeration value range allowed by the business value domain constraint; Third level: If the second level check is passed, determine whether the source data value is within the second enumeration value range allowed by the standard value domain constraint.

5. The method for standard adaptive execution based on data standard management according to claim 3, characterized in that: The abnormal snapshot includes at least one or more of the following information: abnormal data value and its original metadata, violated specific value range constraints, associated mapping relationship labels, and abnormal verification level identifiers.

6. A standard adaptive execution method based on data standard management according to claim 4, characterized in that: The method further comprises: When it is determined that the source data value does not meet the type matching condition corresponding to the data type constraint, the preset conversion rule is executed to convert the source data type corresponding to the source data value into the target field type; the preset conversion rule includes at least one or more of the following: string to numeric conversion, numeric to string conversion, date format standardization conversion, and Boolean to integer mapping conversion.

7. The method for standard adaptive execution based on data standard management according to claim 1, characterized in that: Before determining that a data standard attribute change event exists, the method further includes: Monitoring the standard field template and determining whether the attributes of the standard field template are modified; If so, mark the standard set associated with the standard field model and generate the data standard attribute change event.

8. The method for standard adaptive execution based on data standard management according to claim 1, characterized in that: Determine if there are abnormal snapshots that meet the preset conditions, including: Accumulating the number of abnormal snapshots of the abnormal snapshots, and determining the business standard type corresponding to each abnormal snapshot; In a case where the number of abnormal snapshots is greater than a predetermined number value and / or the abnormal snapshots of the selected business standard type exist, it is determined that the preset condition is met.

9. The method for standard adaptive execution based on data standard management according to claim 1, characterized in that: Based on the content of the data standard version change or the root cause classification results of each abnormal snapshot, a corresponding update policy type is determined to perform standardized execution according to the corresponding update policy, specifically including: Determine the update policy type according to the change content or the root cause classification result through policy determination rules; wherein the policy determination rules include: determining the update policy type as a field value conversion script when the change content involves a value range or a data structure, determining the update policy type as a mapping rule update instruction when the change content only involves semantic rules or association relationships, determining the update policy type as a field value conversion script when the root cause classification result is a standard definition defect, and determining the update policy type as a mapping rule update instruction when the root cause classification result is a mapping rule error; In a case where the update strategy type is a field value conversion script, generating and executing the field value conversion script to convert the source data value; In a case where the update strategy type is the mapping rule update instruction, the mapping rule update instruction is generated and the parameters of the preset semantic matching model are adjusted.

10. A standard adaptive execution device based on data standard management, characterized in that: The device is applied to a data standard management system; the device includes: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the standard adaptive execution method based on data standard management as described in any one of claims 1 to 9.

Images