Metadata registration standard-based data semantic automatic sharing method and electronic equipment

Abstract

The invention provides a metadata registration standard-based data semantic automatic sharing method and electronic equipment, and belongs to the technical field of data management. The method comprises the following steps: firstly, collecting sub-domain metadata corresponding to different data domains, carrying out data preprocessing on the collected sub-domain metadata, then extracting shared domain metadata based on the preprocessed sub-domain metadata, constructing a data semantic sharing model, and finally, taking the data semantic sharing model as input, so that the data semantic sharing is realized. And automatic registration of shared domain data elements is realized through a multi-agent system, that is, efficient conversion from a data semantic sharing model to an actual registration result can be realized through cooperative work of a planning agent and an execution agent, and the problems of low efficiency, proneness to errors, insufficient standardization degree and the like faced by traditional manual registration are solved.

Description

Technical Field

[0001] This application relates to the field of data governance technology, and in particular to a data semantic automated sharing method and electronic device based on metadata registration standards. Background Technology

[0002] With the rapid development of information technology, government affairs, healthcare, and other fields are facing an explosive growth in data volume. Big data has become an important force driving social progress and technological innovation. However, in large-scale distributed scenarios, data aggregation and interoperability face enormous challenges, leading to the formation of multiple independent data domains and creating cross-domain data silos. This fragmentation mainly stems from differences in data representation standards and inconsistent model architectures between systems, resulting in semantic fragmentation and inconsistency. Taking the healthcare industry as an example, the same entity may be labeled as "Gender," "Sex," or "patient gender" by different systems. This semantic heterogeneity means that although the data can be obtained, it is difficult to understand and use accurately, creating a "data visible but unusable" dilemma. Therefore, achieving semantic unification and interoperability across data sources while ensuring data accessibility has become a key technical challenge that urgently needs to be overcome in the field of data sharing.

[0003] The International Organization for Standardization (ISO / IEC) has proposed the Metadata Registration Standard (MDR) 11179 to address the issue of metadata semantic interoperability. This standard describes a standardized approach to metadata semantics, defining elements such as data element concepts, value domains, and conceptual domains, thus expressing the semantic meaning of metadata in greater detail. MDR not only supports precise modeling of values ​​and their meanings but also establishes relationships between semantic elements in a shared domain, achieving a higher degree of reusability and consistency at the data element concept and conceptual domain levels, fully meeting the needs of semantic sharing and integration in complex scenarios. However, it relies on manual registration by operators, resulting in significant problems of low efficiency and high error rates. Summary of the Invention

[0004] To address at least one of the aforementioned problems, this application proposes a data semantic automated sharing method and electronic device based on a metadata registration standard. The aim is to construct a data semantic sharing model and achieve automated registration of shared domain data elements based on a multi-agent system. Compared to manual registration, this method significantly reduces registration time and fully complies with the MDR specification, ensuring both registration accuracy and compliance.

[0005] To achieve the above objectives, a first aspect of this application proposes a method for automated data semantic sharing based on a metadata registration standard, the method comprising: Collect the subdomain metadata corresponding to each different data domain, and perform data preprocessing on the collected subdomain metadata; Based on the preprocessed subdomain metadata, shared domain metadata is extracted to construct a data semantic sharing model, wherein the shared domain metadata includes ontology class, attributes, enumerable values ​​and their meanings; Using the data semantic sharing model as input, the automated registration of shared domain data elements is achieved through a multi-agent system. The multi-agent system includes a planning agent and an execution agent. The planning agent generates a structured registration plan that conforms to the metadata registration standard specification based on the shared domain metadata provided by the data semantic sharing model, so as to clarify the order and dependencies of the registration steps. The execution agent executes each step in the structured registration plan to complete the automated registration of shared domain data elements.

[0006] In one embodiment of this application, the step of collecting subdomain metadata corresponding to each different data domain and preprocessing the collected subdomain metadata includes: Determine the target data domain range and collect the subdomain metadata corresponding to each data domain within the target data domain range; The collected subdomain metadata is formatted to ensure that the subdomain metadata format is consistent; A multi-dimensional data quality verification algorithm is used to detect and calculate the quality score of the normalized subdomain metadata in order to select qualified subdomain metadata. A comprehensive semantic matching method based on multiple similarity algorithms is used to perform semantic deduplication on the selected qualified subdomain metadata. The number and category distribution of metadata for each subdomain after statistical semantic deduplication are used to form a subdomain metadata statistical report.

[0007] In one embodiment of this application, the step of using a multi-dimensional data quality verification algorithm to detect the normalized subdomain metadata and calculate a quality score to filter out qualified subdomain metadata includes: The normalized subdomain metadata is subjected to basic integrity checks, format consistency checks, attribute name rationality checks, and outlier detection. Customize the deduction rules and calculate the quality score of each subdomain metadata based on the detection results and the deduction rules; Subdomain metadata with a quality score greater than or equal to a set threshold is selected as qualified subdomain metadata.

[0008] In one embodiment of this application, the step of using a comprehensive semantic matching method based on weighted multiple similarity algorithms to perform semantic deduplication on the selected qualified subdomain metadata includes: For the selected qualified subdomain metadata, calculate the edit distance similarity, Jaccard similarity, synonym expansion similarity, and structural similarity between the metadata of each subdomain. Assign corresponding weight coefficients to the edit distance similarity, the Jaccard similarity, the synonym expansion similarity, and the structural similarity; Based on the weight coefficients corresponding to the edit distance similarity, the Jaccard similarity, the synonym expansion similarity, and the structural similarity, the edit distance similarity, the Jaccard similarity, the synonym expansion similarity, and the structural similarity corresponding to each subdomain metadata are weighted and summed to obtain the comprehensive similarity. Based on the comprehensive similarity, semantic deduplication is performed on the selected qualified subdomain metadata.

[0009] In one embodiment of this application, the step of extracting shared domain metadata based on the preprocessed subdomain metadata and constructing a data semantic sharing model includes: Based on the preprocessed subdomain metadata, and combined with domain business characteristics and data semantic relationships, a data semantic sharing model framework is constructed to clarify the business dimensions that the data semantic sharing model needs to cover. Based on the data semantic sharing model framework, ontology classes are extracted from the preprocessed subdomain metadata; Based on the field information in the subdomain metadata, match the corresponding attributes for each ontology class; Determine the enumerable values ​​and meanings corresponding to the attribute; Semantic consistency checks are performed on the extracted ontology classes, attributes, enumerable values, and their meanings. The extracted metadata is then filtered and integrated to obtain shared domain metadata, thereby constructing a data semantic sharing model.

[0010] In one embodiment of this application, after constructing the data semantic sharing model, the method further includes: The performance of the constructed data semantic sharing model is evaluated based on three metrics: coverage, reuse rate, and convergence rate.

[0011] In one embodiment of this application, the planning agent generates a structured registration plan conforming to the metadata registration standard specification based on the shared domain metadata provided by the data semantic sharing model, so as to clarify the order and dependencies of the registration steps, including: Obtain the shared domain metadata provided by the data semantic sharing model; Prompt words are constructed based on a symbolic reasoning strategy to require the planning agent to transform the registered task into a symbolic plan sequence based on the programming language syntax; A few-sample prompting strategy is adopted to embed registration cases as contextual references in the prompt words, so as to guide the planning agent to generate a structured registration plan that conforms to the metadata registration standard specification and contains specific tool call instructions based on the shared domain metadata, so as to clarify the order and dependencies of the registration steps.

[0012] In one embodiment of this application, the executing agent performs various steps in the structured registration plan to complete the automated registration of shared domain data elements, including: Receive the structured registration plan generated by the planning agent; The execution agent uses an iterative "reasoning-action-observation" mode to execute the structured registration plan. In the reasoning phase, the execution agent analyzes the intent of the current step in the structured registration plan. In the action phase, it calls a custom toolset to interact with the graph database. In the observation phase, it receives execution feedback and dynamically adjusts subsequent operations based on the feedback results. If a verification error occurs during execution, the executing agent will automatically trigger the exception correction logic, adjust the parameters, and re-execute the current step.

[0013] In one embodiment of this application, the custom toolset includes element registration tools, symbol processing tools, relationship referencing tools, and query and verification tools, wherein... The element registration tool is used to create object classes, attributes, conceptual domains, data element concepts, value domains, and data element nodes; The symbol processing tools are used to parse unstructured enumeration value strings and convert them into a standardized key-value pair format; The relational reference class tool is used to establish associations between semantic nodes, including the association between value domain and conceptual domain and the association between data element and data element concept; The query and verification tool is used to search for the existence of nodes with the same name before registration to avoid duplicate registration, and to verify the compliance of the data format.

[0014] To achieve the above objectives, a second aspect of this application provides an electronic device, comprising: One or more processors; A memory for storing one or more computer programs that, when executed by one or more processors, cause the processors to perform the method as described in any embodiment of this application.

[0015] In the technical solution provided in this application embodiment, firstly, the metadata of the subdomains corresponding to different data domains is collected, and the collected subdomain metadata is preprocessed to ensure the accuracy and efficiency of subsequent shared domain metadata extraction and automated registration. Then, based on the preprocessed subdomain metadata, shared domain metadata is extracted, and a data semantic sharing model is constructed. This model, through ontology class extraction, attribute matching, and value domain standardization, can unify heterogeneous representations of the same entity in different subdomains into a standardized ontology class system, achieving automatic alignment of semantic heterogeneity between different data domains and eliminating semantic dispersion and inconsistency problems. The constructed data semantic sharing model can also significantly reduce the time for semantic parsing and conflict handling in the subsequent registration process. Finally, using the data semantic sharing model as input, the automated registration of shared domain data elements is achieved through a multi-agent system. That is, through the collaborative work of the planning agent and the execution agent, efficient transformation from the data semantic sharing model to the actual registration result can be achieved, solving the problems of low efficiency, error-proneness, and insufficient standardization faced by traditional manual registration.

[0016] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the principles of this application.

[0018] Figure 1 This is a flowchart of a data semantic automated sharing method based on metadata registration standards provided in this application embodiment.

[0019] Figure 2 This is a flowchart illustrating the steps of collecting subdomain metadata corresponding to different data domains and performing data preprocessing on the collected subdomain metadata, as provided in this application embodiment.

[0020] Figure 3 This is a flowchart illustrating the steps of using a multi-dimensional data quality verification algorithm to detect and calculate quality scores for the normalized subdomain metadata in order to select qualified subdomain metadata, as provided in this application embodiment.

[0021] Figure 4 This is a flowchart illustrating the steps of semantic deduplication of selected qualified subdomain metadata using a comprehensive semantic matching method based on multiple similarity algorithms, as provided in this application embodiment.

[0022] Figure 5 This is a flowchart illustrating the steps of extracting shared domain metadata from preprocessed subdomain metadata and constructing a data semantic sharing model, as provided in this application embodiment.

[0023] Figure 6 This is a data semantic sharing model architecture diagram provided in the embodiments of this application.

[0024] Figure 7 This is a conceptual data model diagram based on the MDR specification provided in the embodiments of this application.

[0025] Figure 8 The flowchart provided in this application embodiment is a step flowchart that clarifies the order and dependencies of registration steps by generating a structured registration plan that conforms to the metadata registration standard specification based on the shared domain metadata provided by the data semantic sharing model.

[0026] Figure 9 This is a flowchart illustrating the steps of an execution agent in executing a structured registration plan to complete the automated registration of shared domain data elements, as provided in this embodiment of the application.

[0027] Figure 10 This is a registration flowchart illustrating the automated registration of shared domain data elements in a multi-agent system provided in this application embodiment.

[0028] Figure 11 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0029] To make the objectives, implementation methods, and advantages of this application clearer, exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these exemplary embodiments are provided to make the description of this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. It should be noted that the brief descriptions of terminology in this application are merely for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0030] In the description of this application, it should be understood that the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more features.

[0031] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0032] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0033] With the rapid development of information technology, fields such as government affairs and medicine are facing an explosive growth in data volume. Big data, as an emerging form of data, has gradually become an important force driving social progress and technological innovation.

[0034] However, in large-scale distributed scenarios, data aggregation and interoperability face significant challenges due to technical and management constraints, leading to the formation of multiple independent data domains and creating cross-domain data silos. Different data domains suffer from inconsistent and fragmented semantics due to variations in representation standards and data models, making semantic sharing difficult. For example, in the medical field, different data domains may use different expressions (such as "Gender," "Sex," and "patient gender") to refer to the same entity. Without addressing semantic aggregation and interoperability issues, the data may be accessible but difficult to understand, resulting in a situation of "accessible but incomprehensible." Therefore, achieving semantic alignment and interoperability between different data sources while ensuring data accessibility has become a crucial issue in data sharing.

[0035] While data governance technologies can partially address semantic alignment issues from a technical perspective, their reliance on centralized collection and management makes them difficult to meet the cost and timeliness requirements of large-scale scenarios. For example, data platforms solve the data silo problem by building an organization-level data management and distribution platform, but the high cost and low flexibility of their centralized governance limit their application in dynamic scaling and real-time needs.

[0036] Ontology-Based Data Access (OBDA), a common data integration paradigm, typically uses ontology construction techniques to semantically model the shared layer. Using the ontology as an intermediary language, it provides domain concepts as a semantic sharing layer for the underlying heterogeneous data sources. However, ontologies still have some shortcomings in terms of the richness and detail of semantic expression. For example, traditional ontologies struggle to fully represent different representations of attribute values ​​and their corresponding meanings, often limiting the depth and breadth of semantic expression, thus resulting in limitations in semantic richness and consistency.

[0037] The International Organization for Standardization (ISO / IEC) has proposed the Metadata Registration Standard (MDR) 11179 to address the issue of metadata semantic interoperability. This standard describes a standardized approach to metadata semantics, defining elements such as data element concepts, value domains, and conceptual domains, thus expressing the semantic meaning of metadata in greater detail. MDR not only supports precise modeling of values ​​and their meanings but also establishes relationships between semantic elements in shared domains, achieving a higher degree of reusability and consistency at the data element concept and conceptual domain levels, fully meeting the needs of semantic sharing and integration in complex scenarios. However, it relies on manual registration by operators, resulting in significant problems of low efficiency and high error rates.

[0038] Based on this, this application proposes a data semantic automated sharing method based on metadata registration standards, which transforms the traditional registration process that relies on human experience into a standardized operation that can be parsed by machines, thus completely solving the problems of low efficiency and error-proneness of manual operation.

[0039] Reference Figure 1 , Figure 1 This is a flowchart of a data semantic automated sharing method based on metadata registration standards provided in this application embodiment, including but not limited to steps S110 to S130.

[0040] Step S110: Collect the subdomain metadata corresponding to each different data domain, and perform data preprocessing on the collected subdomain metadata.

[0041] In this step, we collect the metadata of the subdomains corresponding to each different data domain, and perform data preprocessing on the collected subdomain metadata to provide a high-quality data foundation for subsequent processing.

[0042] Specifically, refer to Figure 2 , Figure 2 This is a flowchart of the steps for collecting subdomain metadata corresponding to different data domains and performing data preprocessing on the collected subdomain metadata, including but not limited to steps S210 to S250, provided in the embodiments of this application.

[0043] Step S210: Determine the target data domain range and collect the subdomain metadata corresponding to each data domain within the target data domain range.

[0044] In this step, the target data domain scope is determined (such as sub-domains in the fields of government affairs and healthcare). Metadata of the relevant sub-domains (which must include core fields such as ontology class, attributes, values, and value meanings) is collected through sub-domain system interfaces, structured document import, or manual data entry, and recorded as the initial sub-domain metadata dataset to ensure that the dataset covers the core business scenarios of the sub-domain.

[0045] For example, metadata for five typical medical subdomains (breast cancer NCT00433589 endocrine therapy subdomain, colorectal cancer NCT00303628 non-protocol treatment subdomain, dbGaP phs000487 prostate cancer risk subdomain, endometrial cancer phs000841 tyrosine kinase mutation subdomain, and pancreatic cancer NCT00058149 quality of life assessment subdomain) can be cleaned and organized to provide a foundation for building a shared model. Specifically, subdomain metadata for each medical subdomain can be collected through a combination of subdomain system interfaces (such as cancer clinical trial database interfaces and dbGaP sample library interfaces) and manual data entry, and stored as a structured document with fields including "ontology class, attribute, value, value meaning, code, data type, attribute description, and source file".

[0046] Step S220: Standardize the format of the collected subdomain metadata to ensure that the subdomain metadata format is uniform.

[0047] In this step, the collected metadata of each subdomain is standardized in format, and the field naming rules and data type description specifications are unified. At the same time, data quality verification algorithms are used to perform integrity verification and consistency checks, and invalid metadata containing problems such as missing fields and abnormal formats is removed, thereby ensuring the basic validity and usability of the subdomain metadata.

[0048] For example, following the previous example, a unified format of "domain terminology + core meaning" can be adopted, such as labeling "OFS" as "Ovarian Function Suppression"; dates can be uniformly formatted as "YYYY-MM-DD" (e.g., "NoteDate" is changed from "05 / 12 / 2023" to "2023-05-12"); Boolean fields can be uniformly represented as "True / False" (e.g., "Non-protocol Immunotherapy" is changed from "Yes / No" to "True / False"); 18 missing fields can be removed (e.g., "Sample ID" is not filled in), and finally 540 valid medical subdomain metadata records are retained.

[0049] Step S230: The normalized subdomain metadata is detected and a quality score is calculated using a multi-dimensional data quality verification algorithm to select qualified subdomain metadata.

[0050] In this step, a multi-dimensional data quality verification algorithm is used to perform multi-dimensional detection and calculate a quality score, so that qualified subdomain metadata can be screened based on the calculated quality score.

[0051] Specifically, refer to Figure 3 , Figure 3This application provides a flowchart of the steps for detecting and calculating quality scores of normalized subdomain metadata using a multi-dimensional data quality verification algorithm to select qualified subdomain metadata, including but not limited to steps S310 to S330.

[0052] Step S310: Perform basic integrity checks, format consistency checks, attribute name rationality checks, and outlier checks on the normalized subdomain metadata; Step S320: Define the deduction rules and calculate the quality score of each subdomain metadata based on the detection results and the deduction rules; Step S330: Select subdomain metadata with a quality score greater than or equal to a set threshold as qualified subdomain metadata.

[0053] In this step, the normalized subdomain metadata is first subjected to basic integrity checks, format consistency checks, attribute name rationality checks, and outlier checks.

[0054] Among these, basic integrity testing is the primary step in metadata quality verification, and its core lies in ensuring the structural integrity and semantic consistency of the data. Specifically, it includes two key steps: First, perform non-empty validation on required fields such as "ontology class" and "attribute" to prevent the loss of core information; second, perform a correspondence check on the "value" and "value meaning" fields, that is, after splitting the data by the same delimiter (such as semicolon or comma), strictly compare whether the number of items in the two groups matches (for example, if "value" is split into 3 items, "value meaning" must also have 3 items), thereby ensuring that each value can accurately map to its business meaning, avoiding subsequent semantic parsing errors or data unavailability due to missing fields or misaligned mappings.

[0055] The core objective of format consistency detection is to ensure the standardized and uniform representation of data through automated detection methods such as regular expressions. The specific implementation involves three dimensions of detection: First, pattern matching identifies non-standard characters (such as control characters, illegal symbols, etc.) to eliminate the risk of data contamination; second, regular expressions are used to detect formatting issues caused by consecutive punctuation marks (such as ";;;" or ",", etc.); and finally, a systematic check is performed to address the mixing of English and Chinese delimiters (such as the simultaneous appearance of ";" and ";"), forcing uniformity to a single standard delimiter. This detection mechanism, through multi-dimensional pattern recognition, constructs a complete format specification system from the character level, symbol level, to the language level, providing a structured data foundation for subsequent semantic analysis.

[0056] The core objective of attribute name rationality checking is to ensure the standardization and semantic clarity of attribute naming through a dual constraint mechanism. First, a length limit (attribute name ≤ 50 characters) avoids verbose names, ensuring system processing efficiency and storage optimization. Second, validation using common type keywords (such as standard terms like "time" and "address") ensures that attribute naming conforms to domain specifications, avoiding semantic confusion caused by arbitrary naming. This checking mechanism, through the synergy of structural constraints and semantic validation, technically guarantees the accurate identification and unambiguous understanding of metadata during subsequent semantic parsing, registration, and cross-domain sharing processes.

[0057] The core objective of outlier detection is to identify non-standard values ​​in the "data type" field through standard type comparison. In practice, the system pre-defines a set of standard types (such as "string," "number," and "date") and uses an exact matching algorithm to verify whether the value of each "data type" field conforms to this standard set. For detected non-standard types (such as "text" or "numeric," etc.), the system marks them as anomalous data and triggers a correction process. This mechanism ensures type compatibility of metadata during cross-system interactions from a data standardization perspective, avoiding data parsing errors or business logic anomalies caused by ambiguities in type definitions.

[0058] Next, a quantitative assessment and tiered screening of metadata is achieved through customized deduction rules and a quality scoring mechanism. Specifically, a tiered deduction standard is set based on the severity of the detected problems (10 points are deducted for high-severity problems such as missing required fields, 5 points for medium-severity problems, and 2 points for low-severity problems); then, the total deduction is calculated based on the detection results, and the quality score is calculated by combining the maximum possible deduction (total number of records × 10). The formula for calculating the quality score is: The maximum possible deduction is equal to the total number of records × 10 (i.e., a maximum deduction of 10 points per record). After calculating the quality score, qualified subdomain metadata can be filtered based on the quality score. For example, subdomain metadata with a quality score ≥ 80 can be judged as qualified data and directly proceed to the semantic deduplication step. Subdomain metadata with a quality score between 60 and 79 can be judged as data to be repaired. After automatic completion of missing fields by the algorithm module (such as completing the "value meaning" based on the domain dictionary) or manual correction, quality verification is performed again. Subdomain metadata with a quality score < 60 is judged as poor-quality data and is directly removed, not participating in subsequent processes.

[0059] For example, following the previous example, for the aforementioned 540 valid medical subdomain metadata, the basic integrity check detected that 22 subdomain metadata entries had inconsistent groupings of "value" and "value meaning" (e.g., "value" is "0;1;2", "value meaning" is only "no adverse events; mild"). These were marked as medium severity issues, and 5 points were deducted for each entry. There were no missing required fields, and the high severity issue was 0.

[0060] The format consistency check used the regular expression [^\w\s\u4e00-\u9fa5,;,;\-\.] to detect 7 subdomain metadata entries containing the special characters "@" and "#". The check also used [,;,;]{3,} to detect 3 entries containing consecutive punctuation marks ";;;". All were marked as medium-severity issues, with 5 points deducted for each. The format conflict was corrected by unifying the Chinese and English separators and converting 15 metadata entries containing ";" and "," to ";" and ",".

[0061] The attribute name rationality check restricts the attribute name length to ≤50 characters. Two "patient postoperative 30-day complication follow-up records" (58 characters) were found and marked as low-severity issues, with 2 points deducted for each record. The "data type" field was verified, and 10 metadata entries marked as "text" and "numeric" were corrected to the standard "string" and "number" types, which is a medium-severity issue and deducts 5 points per entry.

[0062] Quality scoring and data filtering are calculated using the quality scoring formula: =95.8 points. A total of 510 qualified data entries with scores ≥80 points were ultimately selected, with no data requiring repair or of poor quality.

[0063] Step S240: Use a comprehensive semantic matching method based on multiple similarity algorithms to perform semantic deduplication on the selected qualified subdomain metadata.

[0064] In this step, after selecting qualified subdomain metadata, a comprehensive semantic matching method based on multiple similarity algorithms can be used to perform semantic deduplication on the selected qualified subdomain metadata, so as to eliminate redundant storage and avoid duplicate registration in subsequent shared domain metadata.

[0065] Specifically, refer to Figure 4 , Figure 4 This is a flowchart of the steps for semantic deduplication of selected qualified subdomain metadata using a comprehensive semantic matching method based on multiple similarity algorithms, provided in the embodiments of this application, including but not limited to steps S410 to S440.

[0066] Step S410: For the selected qualified subdomain metadata, calculate the edit distance similarity, Jaccard similarity, synonym expansion similarity and structural similarity between each subdomain metadata. Step S420: Assign corresponding weight coefficients to edit distance similarity, Jaccard similarity, synonym extended similarity, and structural similarity; Step S430: Based on the weight coefficients corresponding to edit distance similarity, Jaccard similarity, synonym expansion similarity and structural similarity, the edit distance similarity, Jaccard similarity, synonym expansion similarity and structural similarity corresponding to each subdomain metadata are weighted and summed to obtain the comprehensive similarity. Step S440: Perform semantic deduplication on the selected qualified subdomain metadata based on comprehensive similarity.

[0067] In this step, for the selected qualified subdomain metadata, the edit distance similarity, Jaccard similarity, synonym expansion similarity, and structural similarity between the metadata of each subdomain are calculated respectively.

[0068] Edit distance similarity is based on the Levenshtein distance principle, measuring the degree of matching between two strings by calculating the minimum number of editing operations (including insertion, deletion, and replacement). Its core principle is that the fewer the number of editing operations, the more similar the strings are. Specifically, the difflib.SequenceMatcher tool is typically used to calculate the similarity ratio between strings (range 0-1), with the formula: edit_similarity = 1 - ... .

[0069] Jaccard similarity is a set theory-based method for calculating text similarity. It measures the semantic relevance of texts by calculating the ratio of the intersection to the union of two word sets. Its core principle is that the more common words a text has, the higher its semantic relevance. Specifically, the text is segmented into word sets, and the formula is: jaccard_similarity = ... In this method, A and B represent the word sets of the two texts, the numerator is the intersection size (number of common words), and the denominator is the union size (total number of words). This method simplifies the traditional TF-IDF calculation, directly utilizes the statistical features of word sets to assess similarity, and avoids complex word frequency weight calculations, making it particularly suitable for short text or keyword matching scenarios.

[0070] Synonym-based similarity is a semantic matching method based on word set expansion. Its core principle is to map the original word set to a richer set of semantic expressions through synonym expansion, thereby capturing lexical relationships that are literally different but semantically related (such as "time" and "date"). Specifically, the original word sets A and B are first expanded using synonyms to form expanded word sets A' and B'. Then, the Jaccard similarity formula is used to calculate the intersection ratio of the expanded word sets: synonym_similarity = The numerator |A'∩B'| represents the number of common words after expansion (including synonyms), and the denominator |A'∪B'| is the total vocabulary size of the expanded word set. This method, through a semantic expansion mechanism, can effectively identify cross-linguistic or synonymous semantic equivalence relationships such as "user name" and "UserName", and has stronger semantic inclusiveness than simple literal matching.

[0071] Structural similarity is a semantic analysis method based on word order matching, specifically designed to evaluate the semantic relevance of compound words (such as "username" and "user name"). Its core principle is that when the word segmentation results of compound words are highly similar in order, their overall semantics are more likely to be similar. In practice, the compound words are first segmented (e.g., "username" is split into ["user", "name"], and "user name" is split into ["user", "name"]), and then a sequence matching algorithm (such as edit distance or longest common subsequence algorithm) is used to calculate word order similarity. This method, by analyzing the structural features of word arrangement, can effectively identify compound words like "username" and "user name" that are highly similar in morpheme composition and order, compensating for the shortcomings of semantic analysis methods based solely on word frequency or bag-of-words models, and providing a more dimensional basis for semantic matching.

[0072] The comprehensive semantic similarity matching algorithm achieves accurate semantic matching and deduplication through multi-dimensional weighted calculations. Its core mechanism includes four key dimensions: Edit distance similarity (weight 30%), which measures literal matching based on the minimum number of string editing operations; Jaccard similarity (weight 30%), which assesses lexical overlap through the ratio of word set intersection and union; Synonym extended similarity (weight 25%), which captures semantic equivalence relationships using a synonym extended word set; and Structural similarity (weight 15%), which calculates word order similarity after word segmentation of compound words, reflecting the structural relevance. Finally, a weighted average formula is used to calculate the comprehensive similarity: comprehensive_similarity = ).in, The weighting coefficients are (edit distance 0.3, Jaccard 0.3, synonyms 0.25, structure 0.15), and si is the corresponding similarity. If the overall similarity is ≥0.95, it is determined to be semantically duplicated metadata, its semantic relationship is marked, and it is merged into a core record; if the overall similarity is <0.85, it is retained as independent metadata. Through this processing, a non-redundant and semantically clear data foundation is provided for subsequent extraction of shared domain metadata.

[0073] For example, continuing from the previous example, for the 510 qualified subdomain metadata entries selected, taking the metadata related to "patient initials" as an example, the edit distance similarity is: the lengths of "Patient Initials" (subdomain 1) and "ParticipantInitials" (subdomain 2) are 15 and 19 respectively, and the minimum number of edit operations is 4. Therefore, edit_similarity = 1- =0.789.

[0074] Jaccard similarity: After word segmentation, the word sets are {Patient, Initials} and {Participant, Initials}, with an intersection size of 1 and a union size of 3. Therefore, jaccard_similarity = =0.333.

[0075] Extended Synonym Similarity: Based on the NCI Medical Dictionary extended term set {Patient, initial, Patient, Initials} and {Participant, initial, Participant, Initials}, the intersection size is 2, and the union size is 4. Therefore, synonym_similarity = =0.5.

[0076] Structural similarity: The word order is "subject-attribute", and the similarity is 0.9.

[0077] Comprehensive similarity: comprehensive_similarity= =0.96. This indicates duplicate metadata that needs to be merged and deduplicated.

[0078] Deduplication results: A total of 77 semantically duplicated subdomain metadata records were identified (overall similarity ≥ 0.95), which were merged into 34 core records. Finally, 433 semantically unique medical subdomain metadata records were selected, laying the foundation for shared domain metadata extraction.

[0079] Step S250: Calculate the quantity and category distribution of metadata for each subdomain after semantic deduplication, and generate a subdomain metadata statistical report.

[0080] In this step, after semantic deduplication, a structured statistical report is generated by statistically analyzing the total amount of metadata in each subdomain, the number of ontology classes, the number of attributes, and the proportion of different data types (such as the distribution ratio of numeric, text, and date metadata). This clarifies the core content (such as key business entities and attribute characteristics) and coverage (such as the completeness of business scenario coverage) of the data in each subdomain, providing data support for subsequent steps in extracting shared domain metadata. This process ensures that the construction of the data semantic sharing model has a sufficient data foundation through quantitative analysis, while also providing traceable statistical evidence for cross-domain semantic alignment.

[0081] For example, following the previous example, the quantity and category distribution of metadata for each subdomain after semantic deduplication are statistically analyzed, resulting in the subdomain metadata statistical report shown in Table 1 below. The number of valid metadata entries is greater than the number of semantically unique entries because some subdomains contain metadata with different field names but the same semantic meaning (such as "patient height" and "Height"). After semantic deduplication, these are grouped into the same semantic entry, ultimately resulting in 433 semantically unique medical subdomain metadata entries.

[0082] Table 1

[0083] Step S120: Extract shared domain metadata based on the preprocessed subdomain metadata and construct a data semantic sharing model. The shared domain metadata includes ontology classes, attributes, enumerable values ​​and their meanings.

[0084] In this step, after preprocessing the collected subdomain metadata, shared domain metadata can be extracted based on the preprocessed subdomain metadata to construct a data semantic sharing model. The shared domain metadata includes ontology classes, attributes, enumerable values, and the meaning of those values.

[0085] Specifically, refer to Figure 5 , Figure 5 This is a flowchart of the steps for extracting shared domain metadata based on preprocessed subdomain metadata and constructing a data semantic sharing model, provided in an embodiment of this application, including but not limited to steps S510 to S550.

[0086] Step S510: Based on the preprocessed subdomain metadata, and combined with domain business characteristics and data semantic relationships, construct a data semantic sharing model framework to clarify the business dimensions that the data semantic sharing model needs to cover.

[0087] In this step, based on the high-quality subdomain metadata dataset preprocessed in step S110, the construction of the data semantic sharing model first requires establishing a model framework and clarifying the core business dimensions by combining the business characteristics of the target domain with the semantic relationships of the data. This process analyzes the business scenarios and data characteristics of each subdomain to determine the key business entities (such as "approval items" in the government sector or "patient information" in the medical field) and their relationships that the model needs to cover, providing structured guidance for subsequent ontology class extraction and attribute matching. The core objective of the framework construction is to achieve cross-domain data interoperability through standardized semantic expressions, while ensuring that the model can comprehensively reflect the business needs and data semantic characteristics of the target domain, laying the foundation for subsequent automated registration and semantic sharing.

[0088] For example, following the aforementioned example and considering the business characteristics of the medical field (patient information management, cancer treatment monitoring, sample collection, adverse event reporting, and clinical trial management), the core business dimensions of the shared model are determined as follows: Patient basic information dimension: covering patient identification and demographic characteristics (gender, age, race); Cancer treatment dimension: covering treatment plan (contracted / non-contracted treatment), treatment time, and treatment intention (adjuvant therapy / palliative therapy); Sample and trial dimension: including sample ID, sample type (tumor / normal), clinical trial period, and adverse event classification; Compliance and consent dimension: including informed consent date and sample use consent status.

[0089] Step S520: Based on the data semantic sharing model framework, extract ontology classes from the preprocessed subdomain metadata.

[0090] This step first involves comprehensively summarizing the core entity concepts involved in the metadata of each subdomain to form an entity concept set. Then, a semantic similarity matching algorithm is used to filter out entities common across subdomains (such as identifying common entities like "patient information" and "treatment plans" in the medical field). These are formally defined as ontology classes in the data semantic sharing model, ensuring that the ontology class system can comprehensively cover the core business entities of each subdomain and avoid the omission of key entity concepts due to semantic heterogeneity or differences in expression. This process, through multi-dimensional semantic matching and universality filtering, constructs a structured foundation for cross-domain semantic alignment.

[0091] For example, following the previous example, we summarize the core entity concepts of 5 subdomains, filter out common entities across subdomains, and define them as ontology classes of the medical shared model, resulting in a total of 6 core ontology classes: Patient, Cancer Treatment, Sample Information, Adverse Event, Clinical Trial Period, and Informed Consent.

[0092] Step S530: Based on the field information in the subdomain metadata, match the corresponding attributes for each ontology class.

[0093] In this step, when matching corresponding attributes for each ontology class, the first step is to extract the set of attributes associated with the ontology class based on the field information in the subdomain metadata. Then, a semantic similarity matching algorithm is used to unify the naming of heterogeneous attributes (e.g., merging "User Name" and "User Name" into the standardized expression "User Name"), ensuring that attribute naming retains both the original semantics and cross-domain universality. This process, through attribute-ontology class association analysis and multi-dimensional semantic matching, constructs a unified attribute system that supports semantic interaction across different subdomains, laying a structured foundation for semantic interoperability in the data sharing model.

[0094] For example, following the previous example, to address the ambiguity of cross-subdomain attributes, a common attribute is matched for each ontology class. For example, the time attribute "Treatment Begin Date" (subdomain 1) and "Projected Start Date of Treatment" (subdomain 5) are unified as "Treatment Projected Begin Date", with the attribute description being "Expected Treatment Start Date (format YYYY-MM-DD)"; the numeric attribute "Patient Weight" (subdomain 3) and "Weight" (subdomain 4) are unified as "PatientWeight", with the value type being "float", and the unit being uniformly labeled in the "Value Meaning" field.

[0095] Step S540: Determine the enumerable values ​​and meanings of the corresponding attributes.

[0096] In this step, for attributes with a fixed value range, the system integrates all possible values ​​of that attribute in each subdomain. Semantic analysis clarifies the business meaning of each enumerable value, establishing a strict one-to-one correspondence between values ​​and semantics. Specifically, it first collects all values ​​for the attribute from each subdomain (e.g., different encoding forms of the "gender" attribute such as "male / female" or "1 / 0"). Then, a semantic mapping algorithm (e.g., similarity calculation based on synonym expansion) identifies the equivalence of different encodings, ultimately forming standardized value-meaning pairs (e.g., "1=male", "0=female"). This process, through cross-domain value integration and semantic association verification, ensures that the value domain definition in the data semantic sharing model comprehensively covers business scenarios and possesses cross-system interoperability, providing a standardized value domain foundation for subsequent automated registration and semantic interaction.

[0097] For example, following the previous example, for attributes with a fixed range of values ​​(such as "Adverse Event Severity Grade"), the values ​​of each subdomain are integrated (subdomain 1 "0; 1; 2; 3; 4; 5", subdomain 2 "No adverse events; Mild; Moderate; Severe; Life-threatening; Death related to adverse events", subdomain 3 "NAE; M; Mod; Sev; LT; DR-AE"), and the meaning of the values ​​is clarified (No adverse events; Mild; Moderate; Severe; Life-threatening; Death related to adverse events).

[0098] Step S550: Perform semantic consistency verification on the extracted ontology classes, attributes, enumerable values ​​and their meanings, filter and integrate the shared domain metadata to construct a data semantic sharing model.

[0099] A comprehensive semantic consistency check is performed on the extracted ontology classes, attributes, enumerable values, and their meanings. Through cross-subdomain comparative analysis, semantic conflicts are identified between the same attributes and values ​​from different sources (e.g., equivalence verification of "Male / 1" and "Male / M" for the "gender" attribute). Finally, through conflict resolution and standardized filtering, shared domain metadata that meets cross-domain sharing requirements is integrated, completing the construction of a domain data semantic sharing model. This process, through multi-dimensional semantic matching and conflict verification mechanisms, ensures that the model retains the business characteristics of each subdomain while achieving cross-system semantic interoperability, providing a standardized and unambiguous semantic foundation for data sharing. The constructed data semantic sharing model can be referenced... Figure 6 , Figure 6 This is a data semantic sharing model architecture diagram provided in an embodiment of this application. For example... Figure 6 As shown, the data semantic sharing model is divided into three layers from top to bottom: the sharing layer, the mapping layer, and the data layer. Each layer works collaboratively through registration and mapping. The sharing layer constructs a lightweight conceptual data model based on the improved MDR specification (see...). Figure 7The system integrates six core registration elements: object classes, attributes, data element concepts, concept domains, value domains, and data elements. While maintaining semantic integrity, it simplifies the registration classes in a business-friendly manner (e.g., merging "data element concepts" and "data elements"), reducing registration complexity. The mapping layer, based on the improved MFI-10 specification, designs a ReSSI mapping registration meta-model, establishing semantic association channels between subdomains and shared domains. It clarifies the correspondence between subdomain data elements and shared domain data elements through mapping scripts (supporting schema-level and instance-level mapping) and defines the mapping rules between the source set (subdomain data elements) and the target set (shared domain data elements). The data layer, as the basic storage layer, manages the field names and values ​​of each subdomain's metadata, providing structured input for semantic registration and mapping. The three layers achieve a closed loop through bidirectional registration-mapping collaboration: the shared layer defines a standardized semantic model, the mapping layer establishes cross-domain association channels, and the data layer supports actual data interaction, together forming a complete semantic sharing and interoperability system.

[0100] For example, following the previous example, after semantic consistency verification and domain expert review, 433 medical shared domain metadata items conforming to the ISO / IEC 11179 (MDR) standard were finally selected and integrated. The mapping example between the core shared domain metadata and subdomains is shown in Table 2. There may be several semantically equivalent data items between shared domain data elements and subdomain metadata.

[0101] Table 2

[0102] In some embodiments, after the data semantic sharing model is constructed, its performance can be evaluated based on three metrics: coverage, reuse rate, and convergence rate.

[0103] Coverage (P) is a core indicator for evaluating the completeness of the data sharing model registry. It is defined as the ratio of the total number of data elements (M) in the MDR registry to the number of semantically unique metadata elements (S) in the subdomain (P = M / S), specifically used to measure the registry's ability to cover subdomain data. When P = 1, it indicates that the registry data elements can completely cover all data items in the subdomain, ensuring that each subdomain data item (such as "patient ID" or "transaction time") can find its corresponding data element in the registry. In practical applications, medical data governance cases show that improving coverage can increase data query accuracy by more than 35%. The numerator M represents the total number of registry data elements, reflecting the theoretical capacity of the model; the denominator S is the number of deduplicated metadata elements in the subdomain, representing the actual scale of business needs. The ratio P directly reflects the degree to which the model meets business needs; for example, in the financial sector, a P ≥ 0.95 is required for the construction of a data platform to meet the standards.

[0104] Reuse rate is used to quantify the ability of a data element in the registry to reuse semantically equivalent data items in its subdomains. Its calculation formula is as follows: Where m represents the number of reused data elements, and M is the total number of data elements in the MDR registry. This metric directly reflects the resource utilization efficiency and management effectiveness of the model by statistically analyzing the proportion of data elements shared by multiple subdomains: the higher the reuse rate, the stronger the model's ability to integrate semantically equivalent data items, which can reduce redundant registrations (such as avoiding duplicate registrations of "User Name" and "User Name") and reduce subsequent maintenance costs.

[0105] Convergence ratio (H) is a core metric for evaluating the level of abstraction in a data sharing model registry. It is defined as the number of mappings from each data element in the shared domain registry to subdomain metadata. The sum of ) and the total number of data elements ( The ratio of 1 / 2 to 1 / 3 is specifically used to quantify a model's ability to semantically integrate heterogeneous data. Its calculation formula is: ,in, Indicates the first The number of mappings from data elements to subdomain metadata. This represents the total number of data elements in the shared domain registry. This indicator directly reflects the abstraction strength of data elements through the statistics of mapping numbers: the more semantically equivalent data items a data element corresponds to in its subdomains (such as "User ID" being reused by multiple subdomains), the stronger its aggregation ability, indicating that the data element has a more significant ability to abstract common features across domains.

[0106] The above indicators (including coverage, reuse rate, and convergence rate) can be used to assess the semantic integrity of the data sharing model under the normalized framework and the degree of abstraction of the shared domain registry, and to evaluate the quality of automatically registering shared domain metadata using a large model.

[0107] For example, following the previous example, the model was pre-evaluated based on three metrics: coverage (P), reuse (D), and convergence (H). The results showed that the shared domain metadata had a coverage rate of P=1 (complete coverage) for 433 semantically unique subdomain metadata, 186 shared domain metadata were reused by two or more subdomains (reuse rate D=0.43), and a single shared domain metadata mapped an average of 1.5 subdomain metadata (convergence rate H=0.36), verifying that the model meets the cross-domain sharing requirements.

[0108] Step S130: Using the data semantic sharing model as input, the shared domain data elements are automatically registered through a multi-agent system. The multi-agent system includes a planning agent and an execution agent. The planning agent generates a structured registration plan that conforms to the metadata registration standard based on the shared domain metadata provided by the data semantic sharing model, so as to clarify the order and dependencies of the registration steps. The execution agent executes each step in the structured registration plan to complete the automatic registration of the shared domain data elements.

[0109] In this step, after constructing the data semantic sharing model, a multi-agent system is used as input to automate the registration of shared domain metadata. This system employs a two-layer collaborative architecture of planning agents and execution agents. The planning agent, based on the shared domain metadata (ontology classes, attributes, enumerable values, and their meanings) provided by the data semantic sharing model, generates a structured registration plan conforming to metadata registration standards such as ISO / IEC 11179, clearly defining the sequence and dependencies of registration steps (e.g., the process constraint of ontology class registration → attribute registration → value domain registration). The execution agent strictly follows this plan, performing the registration operation step-by-step using an atomic toolset to ensure standardized registration of each shared domain metadata and its semantic associations. This process, through the agent's planning-execution closed loop, transforms the semantic model into an operable registration task, achieving automated transformation from semantic definition to actual registration. Simultaneously, the execution results are monitored in real-time through the MDR specification verification interface to ensure the accuracy and compliance of the registration.

[0110] Specifically, refer to Figure 8 , Figure 8 The flowchart provided in this application embodiment is a step flowchart that clarifies the order and dependencies of registration steps, including but not limited to steps S810 to S830. The plan agent generates a structured registration plan that conforms to the metadata registration standard specification based on the shared domain metadata provided by the data semantic sharing model.

[0111] Step S810: Obtain shared domain metadata provided by the data semantic sharing model; Step S820: Construct prompt words based on symbolic reasoning strategy to require the planning agent to convert the registered task into a symbolic plan sequence based on the programming language syntax; Step S830: A few-sample prompting strategy is adopted to embed registration cases in the prompt words as contextual references, so as to guide the planning agent to generate a structured registration plan based on shared domain metadata that conforms to the metadata registration standard specification and contains specific tool call instructions, so as to clarify the order and dependencies of registration steps.

[0112] In this step, the data semantic sharing model constructed in step S120 is first received as input. This model provides structured shared domain metadata, including ontology classes, attributes, and concept domains. Next, prompts are constructed based on a symbolic reasoning strategy to require the planning agent to transform the registration task into a symbolic plan sequence based on programming language syntax. Specifically, the agent's role is explicitly defined as a "metadata registration expert" through system prompts, endowing it with prior knowledge of the ISO / IEC 11179 standard and its core elements. Unlike traditional natural language reasoning, this step mandates the output of a symbolic plan based on programming language syntax (e.g., register_object_class(object_class="patent")) to eliminate ambiguity in natural language. Simultaneously, a few-shot prompting strategy is employed, embedding standard registration cases (e.g., the entire process of "patent legal status") within the prompts as contextual references. This prevents model illusion and overfitting while ensuring the generated registration plan conforms to MDR specifications. This design achieves efficient transformation from a semantic model to a machine-executable registration plan through a three-tiered mechanism of role definition → symbolic constraint → example guidance. The prompt word templates can be found in Table 3 below.

[0113] Table 3

[0114] Reference Figure 9 , Figure 9 This is a flowchart of the steps in the structured registration plan performed by the execution agent in the embodiments of this application to complete the automated registration of shared domain data elements, including but not limited to steps S910 to S930.

[0115] Step S910: Receive the structured registration plan generated by the planning agent; In step S920, the executing agent executes the structured registration plan using an iterative "reasoning-action-observation" mode. In the reasoning phase, the executing agent analyzes the intent of the current step in the structured registration plan, in the action phase calls a custom toolset to interact with the graph database, and in the observation phase, it receives execution feedback and dynamically adjusts subsequent operations based on the feedback results. Step S930: If a verification error occurs during execution, the execution agent will automatically trigger the exception correction logic, adjust the parameters, and re-execute the current step.

[0116] In this step, the executing agent uses an iterative "ReAct" (Re-Act) model to perform the automated registration task. Its core process includes three key stages: In the reasoning stage, the agent parses the symbolic registration plan (e.g., register_object_class("patent")) generated by the planning agent, analyzing the registration intent and parameter requirements of the current step; in the action stage, it calls custom toolsets (e.g., IDGenerator, MDRValidator, etc.) to interact with the graph database, completing specific operations such as data element registration and attribute association; in the observation stage, it receives execution feedback through the MDR specification verification interface. If a verification error is detected, it automatically triggers exception correction logic (e.g., adjusting attribute naming rules or value domain mapping relationships), dynamically adjusts parameters, and re-executes the current step. This closed-loop process, through the intent analysis → tool invocation → feedback verification → dynamic adjustment mechanism, ensures that the registration operation strictly complies with the ISO / IEC 11179 specification requirements, while also possessing error self-correction capabilities, ultimately achieving high-quality automated registration of shared domain data elements.

[0117] The custom toolset includes element registration tools, symbol processing tools, relationship referencing tools, and query and validation tools. The element registration tools are used to create object classes, attributes, conceptual domains, data element concepts, value domains, and data element nodes. The symbol processing tools are used to parse unstructured enumeration value strings and convert them into standardized key-value pair formats. The relationship referencing tools are used to establish associations between semantic nodes, including associations between value domains and conceptual domains, and associations between data elements and data element concepts. The query and validation tools are used to check for duplicate nodes before registration to avoid duplicate registration and to verify the compliance of data formats. Element registration tools (such as `register_object_class`, `register_property`, and `register_value_domain_with_values`) are responsible for the atomic creation of nodes at each level; symbol processing tools (such as `parse_values_and_meanings`) parse unstructured strings (such as "0,1; invalid, valid") into standardized key-value pair formats; relationship referencing tools (such as `create_relationship`) establish cross-node semantic relationships such as "value domain based on concept domain". Simultaneously, the toolset also integrates `IDGenerator` to dynamically generate globally unique identifiers, ensuring the uniqueness and traceability of each node in the knowledge graph. Specifically, the details of the custom toolset can be found in Table 4 below.

[0118] Table 4

[0119] In the automated registration execution process example, the executing agent strictly follows the symbolic registration plan generated by the planning agent step by step. Taking the registration of metadata containing enumerable values ​​such as "patent legal status" as an example, the entire execution process demonstrates the rigor of the ReAct (reasoning-action-observation) model: The agent first enters the thinking phase, recognizing that the input data contains enumerable value features; then, in the action phase, it calls the parse_values_and_meanings tool to format and parse the original string, and confirms the accuracy of the parsing results through the observation phase. Subsequently, the system determines that a value domain needs to be registered. It first uses a query tool to search for nodes with the same name to avoid the risk of duplicate registration. After confirming that there are no conflicts, it calls register_value_domain_with_values ​​to create the value domain node and the corresponding allowed value node in the database. Next, the executing agent recognizes the semantic dependency relationship between the value domain and the concept domain, and calls the create_relationship tool to establish a "BASED_ON" semantic association. Finally, by calling register_data_element, it performs semantic aggregation on the registered ontology classes, attributes, value domains, and other components, completing the final registration of the data element node.

[0120] The entire process also incorporates an exception self-correction mechanism: when an MDR specification verification error is detected during execution, parameter adjustment and retry logic are automatically triggered, and iterative optimization ensures that each registration step complies with the metadata registration standard specification requirements.

[0121] For example, following the previous example, after constructing the data semantic sharing model, the multi-agent Plan-and-Solve (PS) method is used to achieve automated registration of shared domain data elements. The specific implementation is as follows: (3.1) The Plan Agent generates a registration plan: 433 shared domain metadata records (set I={i1,i2,...,i...) 433 The input is the planning agent, which analyzes element dependencies according to the MDR specification (e.g., "register the concept domain first, then register the value domain") and generates a structured registration plan. (See also...) Figure 10 , Figure 10 This is a registration flowchart illustrating the automated registration of shared domain data elements in a multi-agent system provided in this application embodiment. Taking "DEAE severity" as an example, the planned steps are as follows: Step 1: Register the object class "Adverse Event"; Step 2: Register the attribute "AE Severity", associate it with the object class "Adverse Event", and specify the attribute type as "Enumerable"; Step 3: Register the concept domain "AE Severity Concept Domain", described as "a set of grades for defining adverse reactions to cancer treatment"; Step 4: Register the data element concept "AE Severity", and associate it with the object class "Adverse Event" and the attribute "AE Severity"; Step 5: Register the value range "AE Severity Value Range", and simultaneously register the permissible values ​​("0; 1; 2; 3; 4; 5; No adverse events; Mild; Moderate; Severe; Life-threatening; Death related to adverse events; NAE; M; Mod; Sev; LT; DR-AE"). Step 6: Register the meaning of the values ​​in the concept domain ("No adverse events; Mild; Moderate; Severe; Life-threatening; Death related to adverse events"). Step 7: Register the data element "DE AE Severity" and establish its association with "AE Severity Value Range" and "AE Severity Conceptual Domain".

[0122] (3.2) Execution Agent Execution Registration Plan: The execution agent executes the plan using a "think-act-observe" iterative model. "Thinking": Analyze the tools that need to be called in the current step (such as register_concept_domain to register the concept domain, register_value_domain_with_values ​​to register the value domain). "Action": Call the custom toolset and pass in metadata parameters (such as concept domain name, semantic description, enumerable value). For example, when executing Step 3, call the register_concept_domain tool and pass in the "AE severity concept domain" and description information. "Observation": The registration results are checked through the MDR specification verification interface. If it is found that the "Adverse Event" object class has been registered (created when the preceding data element is registered), the object class is automatically reused to avoid redundant operations. (3.3) Exception handling and registration finalization: If a parameter error occurs in a certain step (such as the non-standard format of enumerable values), the executing agent will report back to the planning agent, and the executing agent will correct the plan (such as unifying the value format to "text type") and then re-execute; after all steps are completed, a registration report is generated, and all 433 shared domain metadata entries conform to the MDR specification and are stored in the shared domain registry built by the Neo4j graph database.

[0123] In this embodiment, the large model and agent technology, with its powerful generation and reasoning capabilities, combined with data quality verification algorithms and semantic similarity matching algorithms, can significantly improve the automation level and accuracy of semantic layer analysis, alignment, and data preprocessing. By introducing prompt engineering technology, this framework can generate chain-of-thought prompts, progressively refining the reasoning chain, thereby enhancing the transparency and interpretability of complex semantic alignment tasks. Compared to traditional rule-based or template-based registration methods, the agent based on the PS framework exhibits the core advantages of dynamic adaptability and efficiency: it dynamically generates registration plans using the contextual reasoning capabilities of the large model and adjusts the reasoning process in real time based on contextual information, thereby significantly improving the accuracy and stability of registration. Specifically, the Plan Agent is responsible for generating reasoning steps and execution plans, which are then executed step by step by the Execute Agent, ensuring accurate processing of complex semantic information in dynamically changing scenarios; each step of reasoning is based on the current context and target, and subsequent operations are iteratively optimized to achieve continuous improvement in task execution.

[0124] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0125] Please see Figure 11 , Figure 11 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device includes: The processor 1101 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 1102 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1102 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1102 and is called and executed by the processor 1101. Input / output interface 1103 is used to implement information input and output; The communication interface 1104 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 1105 transmits information between various components of the device (e.g., processor 1101, memory 1102, input / output interface 1103, and communication interface 1105); The processor 1101, memory 1102, input / output interface 1103 and communication interface 1104 are connected to each other within the device via bus 1105.

[0126] Furthermore, to evaluate the efficiency, accuracy, and large-model adaptability of the automatic registration of shared domain metadata provided in this application embodiment, this experiment designed a comparative study of three registration methods: manual, semi-automatic, and fully automatic. The focus was on examining processing time and resource consumption to verify the significant efficiency advantage of automated registration. The experimental results are shown in Table 5 below. The semi-automatic registration employs the core logic of "splitting the processing steps according to whether the metadata contains enumerable values": for metadata without enumerable values, the system automatically completes the registration process; while for metadata containing enumerable values, the manual registration step is retained to ensure accurate processing of key information. Automatic registration registers metadata without enumerable values, while manual registration registers metadata containing enumerable values. Through this differentiated design, the experiment aims to quantitatively analyze the performance of different registration modes in terms of resource utilization and timeliness to verify the registration efficiency of automated registration.

[0127] Table 5

[0128] Meanwhile, this experiment also compared the accuracy of different registration methods from the dimensions of field completeness and format standardization (the results are shown in Table 6 below), verifying the reliability of automated registration. Table 6

[0129] This experiment compares the performance of different large models in the medical data MDR registration task (the comparison results are shown in Table 7 below) to select the model with the best fit.

[0130] Table 7

[0131] Experimental results show that the automated registration method based on multi-agent systems exhibits superior overall performance, significantly improving efficiency compared to manual registration. It effectively reduces manual time and operational costs in batch processing and avoids the randomness issues associated with repetitive manual operations. Semi-automatic registration strikes a reasonable balance between efficiency and manual control, adapting to different scenario requirements. Regarding accuracy, while manual registration performs slightly better overall, fully automated registration achieves accuracy comparable to manual registration. Manual registration is prone to oversights in formatting and association analysis, while fully automated registration occasionally exhibits formatting, semantic, or association judgment errors. These errors are more easily corrected by optimizing the registration logic, resulting in greater controllability. For the large-scale model, GPT-4 was chosen as the optimal choice in terms of registration performance, processing efficiency, and resource utilization. Other intermediate models can be flexibly selected based on the resource budget and requirements of the actual scenario. Furthermore, the synergistic effect of shared domain construction and automated registration effectively solves the semantic dispersion problem of cross-domain data, achieving unified data management and efficient reuse, reducing the long-term maintenance costs of cross-domain data governance, and realizing unified data management while avoiding semantic conflicts.

[0132] To investigate the impact of sample size on automated registration performance, this study designed an ablation experiment with sample size (experimental results are shown in Table 8 below), setting three conditions: 0 samples, 1 sample, and 2 samples. The performance of these conditions in the automated registration task was systematically analyzed. Specifically: Zero samples: The model relies entirely on inference to complete the registration task and does not provide any guidance samples.

[0133] 1 Sample: One manual registration reference sample is provided at the end of the prompt to guide the model's reasoning and generate a registration plan.

[0134] 2 Samples: Two manually registered reference samples are provided at the end of the prompt to guide the model's reasoning and generate a registration plan.

[0135] Table 8

[0136] Experimental results show that different sample numbers significantly impact the model's registration performance. The 1-sample condition yields the best performance, while 0-sample and 2-sample conditions reveal insufficient reasoning ability and overfitting issues, respectively. Under the 0-sample condition, although the model can correctly parse semantic relationships in subdomain data in some tasks, the lack of explicit guidance samples makes it prone to errors when handling complex or ambiguous semantics. This error directly affects the model's recall ability, causing some correct tasks that need registration to be missed. In contrast, the precision drops to 0.87 under the 2-sample condition. Although recall remains at 0.99, the F1 score decreases to 0.93. This phenomenon indicates that under multi-sample guidance, the model suffers from overfitting during reasoning, focusing excessively on the generated format rather than semantic generation, resulting in redundant or repetitive content in the generated registration results. Furthermore, the semantic information provided by multiple samples may conflict or interfere in some cases, negatively impacting task accuracy.

[0137] This study verifies the impact of different prompt word designs (symbolic reasoning and textual reasoning) on ​​the efficiency and accuracy of large-scale model automation tasks in automated registration tasks (experimental results are shown in Table 9 below). Symbolic reasoning uses Python-based coding language to directly generate code that calls specific tools. Textual reasoning uses natural language descriptions to simulate the unstructured expressions in traditional prompt word designs.

[0138] Table 9

[0139] Comparative experiments show that symbolic reasoning prompts significantly outperform text-based prompts in automated registration tasks, fully validating the advantages of structured design in tool invocation and task execution. While text-based prompts achieve a registration accuracy (0.98) close to that of symbolic prompts (0.99), their recall is only 0.55, far lower than the symbolic prompts' 0.99. This results in a mere F1 score of 0.70 for text prompts, demonstrating a significant deficiency in task completion. The core reason for this difference lies in the unstructured nature of text-based prompts. Text-based reasoning relies on natural language descriptions, and its ambiguity can easily lead to parameter passing errors or omissions during parsing and tool invocation. For example, in value domain registration tasks, text prompts often fail to correctly generate complete parameters for the value domain name and enumerable values, resulting in unsuccessful value domain registration.

[0140] Since the registration of data elements depends on the successful registration of value domains, such errors directly cause the failure of subsequent tasks, thus significantly reducing the system's recall rate. Especially in complex tasks with multi-layered dependencies, the semantic ambiguity of text prompts amplifies the uncertainty in tool invocation, significantly increasing the proportion of unregistered tasks. In contrast, symbolic reasoning prompts provide explicit, structured task plans through Python-style tool invocation code. This design ensures the integrity and accuracy of tool invocation parameters, enabling efficient execution of each task step. The advantages of symbolic prompts are particularly significant in complex tasks; their structured nature allows for clear expression of dependencies between tasks, and through dynamically generated plans, step-by-step execution ensures that all dependency conditions are met. Furthermore, the standardized expression of symbolic prompts integrates more tightly with tools, dynamically adapting to different task requirements and reducing ambiguity or execution failures caused by linguistic ambiguity.

[0141] Overall, symbolic reasoning prompts demonstrate superior performance in registration accuracy, recall, and F1 score, reflecting their adaptability and efficiency in complex task scenarios. While textual reasoning prompts perform reasonably well in simple tasks, they reveal deficiencies in parsing capabilities and tool invocation efficiency when handling multi-layered dependencies. Experiments validate that symbolic reasoning prompts, as an efficient and reliable design scheme for automated registration tasks, can significantly improve task accuracy and completion rates, providing crucial technical support for the construction of semantic sharing frameworks. This result further emphasizes the critical role of structured prompt design in automating complex tasks.

[0142] This experiment was conducted on a high-performance server platform, a 2U server configured with the following: two Intel Xeon Gold 6138 processors, each with 20 cores and 80 physical threads, supporting 27.5 MB of L3 cache and 512 GB of DDR4 memory. The operating system was CentOS 7, kernel version 3.10.0-514.16.1.el7.x86_64, and Python 3.8 was used as the primary development language. For storage, Neo4j graph database was used as the underlying data management tool to support efficient relational queries and storage. For model inference, the experiment used the GPT-4 large model, integrating the Plan-and-Execute multi-agent architecture from LangChain.

[0143] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for automated data semantic sharing based on metadata registration standards, characterized in that, The method includes: Collect the subdomain metadata corresponding to each different data domain, and perform data preprocessing on the collected subdomain metadata; Based on the preprocessed subdomain metadata, shared domain metadata is extracted to construct a data semantic sharing model, wherein the shared domain metadata includes ontology class, attributes, enumerable values ​​and their meanings; Using the data semantic sharing model as input, the automated registration of shared domain data elements is achieved through a multi-agent system. The multi-agent system includes a planning agent and an execution agent. The planning agent generates a structured registration plan that conforms to the metadata registration standard specification based on the shared domain metadata provided by the data semantic sharing model, so as to clarify the order and dependencies of the registration steps. The execution agent executes each step in the structured registration plan to complete the automated registration of shared domain data elements.

2. The method according to claim 1, characterized in that, The process of collecting subdomain metadata corresponding to each different data domain and preprocessing the collected subdomain metadata includes: Determine the target data domain range and collect the subdomain metadata corresponding to each data domain within the target data domain range; The collected subdomain metadata is formatted to ensure that the subdomain metadata format is consistent; A multi-dimensional data quality verification algorithm is used to detect and calculate the quality score of the normalized subdomain metadata in order to select qualified subdomain metadata. A comprehensive semantic matching method based on multiple similarity algorithms is used to perform semantic deduplication on the selected qualified subdomain metadata. The number and category distribution of metadata for each subdomain after statistical semantic deduplication are used to form a subdomain metadata statistical report.

3. The method according to claim 2, characterized in that, The process of using a multi-dimensional data quality verification algorithm to detect and calculate quality scores for the normalized subdomain metadata, in order to select qualified subdomain metadata, includes: The normalized subdomain metadata is subjected to basic integrity checks, format consistency checks, attribute name rationality checks, and outlier detection. Customize the deduction rules and calculate the quality score of each subdomain metadata based on the detection results and the deduction rules; Subdomain metadata with a quality score greater than or equal to a set threshold is selected as qualified subdomain metadata.

4. The method according to claim 2, characterized in that, The method employing a comprehensive semantic matching approach based on weighted multiple similarity algorithms to perform semantic deduplication on the selected qualified subdomain metadata includes: For the selected qualified subdomain metadata, calculate the edit distance similarity, Jaccard similarity, synonym expansion similarity, and structural similarity between the metadata of each subdomain. Assign corresponding weight coefficients to the edit distance similarity, the Jaccard similarity, the synonym expansion similarity, and the structural similarity; Based on the weight coefficients corresponding to the edit distance similarity, the Jaccard similarity, the synonym expansion similarity, and the structural similarity, the edit distance similarity, the Jaccard similarity, the synonym expansion similarity, and the structural similarity corresponding to each subdomain metadata are weighted and summed to obtain the comprehensive similarity. Based on the comprehensive similarity, semantic deduplication is performed on the selected qualified subdomain metadata.

5. The method according to claim 1, characterized in that, The step of extracting shared domain metadata based on the preprocessed subdomain metadata and constructing a data semantic sharing model includes: Based on the preprocessed subdomain metadata, and combined with domain business characteristics and data semantic relationships, a data semantic sharing model framework is constructed to clarify the business dimensions that the data semantic sharing model needs to cover. Based on the data semantic sharing model framework, ontology classes are extracted from the preprocessed subdomain metadata; Based on the field information in the subdomain metadata, match the corresponding attributes for each ontology class; Determine the enumerable values ​​and meanings corresponding to the attribute; Semantic consistency checks are performed on the extracted ontology classes, attributes, enumerable values, and their meanings. The extracted metadata is then filtered and integrated to obtain shared domain metadata, thereby constructing a data semantic sharing model.

6. The method according to claim 5, characterized in that, After constructing the data semantic sharing model, the method further includes: The performance of the constructed data semantic sharing model is evaluated based on three metrics: coverage, reuse rate, and convergence rate.

7. The method according to claim 1, characterized in that, The planning agent generates a structured registration plan conforming to the metadata registration standard specification based on the shared domain metadata provided by the data semantic sharing model, in order to clarify the sequence and dependencies of the registration steps, including: Obtain the shared domain metadata provided by the data semantic sharing model; Prompt words are constructed based on a symbolic reasoning strategy to require the planning agent to transform the registered task into a symbolic plan sequence based on the programming language syntax; A few-sample prompting strategy is adopted to embed registration cases as contextual references in the prompt words, so as to guide the planning agent to generate a structured registration plan that conforms to the metadata registration standard specification and contains specific tool call instructions based on the shared domain metadata, so as to clarify the order and dependencies of the registration steps.

8. The method according to claim 1 or 7, characterized in that, The executing agent performs each step in the structured registration plan to complete the automated registration of shared domain data elements, including: Receive the structured registration plan generated by the planning agent; The execution agent uses an iterative "reasoning-action-observation" mode to execute the structured registration plan. In the reasoning phase, the execution agent analyzes the intent of the current step in the structured registration plan. In the action phase, it calls a custom toolset to interact with the graph database. In the observation phase, it receives execution feedback and dynamically adjusts subsequent operations based on the feedback results. If a verification error occurs during execution, the executing agent will automatically trigger the exception correction logic, adjust the parameters, and re-execute the current step.

9. The method according to claim 8, characterized in that, The custom toolset includes element registration tools, symbol processing tools, relationship referencing tools, and query and validation tools, among which... The element registration tool is used to create object classes, attributes, conceptual domains, data element concepts, value domains, and data element nodes; The symbol processing tools are used to parse unstructured enumeration value strings and convert them into a standardized key-value pair format; The relational reference class tool is used to establish associations between semantic nodes, including the association between value domain and conceptual domain and the association between data element and data element concept; The query and verification tool is used to search for the existence of nodes with the same name before registration to avoid duplicate registration, and to verify the compliance of the data format.

10. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more computer programs that, when executed by one or more processors, cause the processors to implement the method as described in any one of claims 1 to 9.

Images