Method and device for automatically generating physical table semantic nodes based on semantic analysis

Abstract

This application proposes a method and apparatus for automatically generating semantic nodes for physical tables based on semantic analysis. The method includes: obtaining at least one unmatched physical table from a target database; performing preprocessing, segmentation, and dictionary matching on the physical table name string of the unmatched physical table to determine at least one semantic root; creating semantic class nodes and associated attribute nodes in a knowledge graph stored in a graph database based on the semantic root; determining existing parent nodes in the knowledge graph and establishing inheritance edge data based on the matching results of the semantic root and pre-stored preset parent class mapping rules; and establishing a first mapping edge data between the semantic class node and the unmatched physical table in the knowledge graph based on the unique metadata identifier of the physical table, and establishing a second mapping edge data between the attribute node and the fields of the unmatched physical table. This application achieves automatic semanticization of non-standard tables through semantic analysis.

Description

Technical Field

[0001] This invention relates to the field of database semanticization and knowledge graph construction technology, and more specifically, to a method and apparatus for automatically generating semantic nodes of physical tables based on semantic analysis. Background Technology

[0002] In related technologies, common database semanticization and knowledge graph construction schemes can only map physical tables that conform to standard naming to the knowledge graph. A large number of non-standard customized tables and extended tables cannot be matched and can only be discarded or used as raw data storage. This results in low knowledge graph coverage, and long-tail business data is outside the semantic network and cannot be effectively retrieved and analyzed. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art or related art.

[0004] Therefore, the first aspect of the present invention proposes a method for automatically generating semantic nodes of physical tables based on semantic analysis.

[0005] A second aspect of the present invention proposes an automatic generation device for physical table semantic nodes based on semantic analysis.

[0006] A third aspect of the present invention proposes another device for automatically generating semantic nodes of physical tables based on semantic analysis.

[0007] In view of this, a first aspect of the present invention provides a method for automatically generating semantic nodes of physical tables based on semantic analysis. The method is executed by a computer and includes: obtaining at least one unmatched physical table in a target database, and a unique metadata identifier for the physical table corresponding to the unmatched physical table. An unmatched physical table refers to a physical table in the target database that has no matching corresponding item after performing mapping matching on the physical tables based on predefined ontology mapping rules; performing preprocessing, segmentation, and dictionary matching processing on the physical table name string of the unmatched physical table to determine at least one semantic root; creating semantic class nodes in a knowledge graph stored in a graph database based on the semantic root, and creating attribute nodes associated with the semantic class nodes based on the field names and field types of the unmatched physical tables; determining existing parent nodes in the knowledge graph according to the matching results of the semantic root and pre-stored preset parent class mapping rules, and establishing inheritance edge data between the semantic class nodes and the parent nodes; and establishing first mapping edge data between the semantic class nodes and the unmatched physical tables in the knowledge graph based on the unique metadata identifier of the physical tables, and establishing second mapping edge data between the attribute nodes and the fields of the unmatched physical tables.

[0008] In this application, the target database is a database system instance storing physical tables to be processed. A physical table is the basic unit of data actually stored in the target database. Predefined ontology mapping rules are a set of mapping logic pre-configured in the computer system, used to establish a correspondence between physical table names and field names and concepts and attributes in the standard ontology library. An unmatched physical table refers to a physical table in the target database that has no matching entries after mapping matching based on the predefined ontology mapping rules. A unique metadata identifier for a physical table is an identifier used within the database system to uniquely identify a physical table, providing the foundation for establishing precise mapping edge data between semantic nodes and physical tables. The physical table name string is an identifier text defined by the developers when the database table is created. Preprocessing operations remove preset affixes to be removed from the physical table name string. Segmentation operations divide the preprocessed string according to preset segmentation rules to obtain multiple independent lexical units. Dictionary matching processing compares the segmented lexical units with a preset semantic dictionary to obtain semantic roots. Semantic roots are semantic lexical units that represent the data object categories in the physical table name string. Graph databases are database systems that use graph data structures as their core storage and query model. Knowledge graphs are the overall graph data structure stored within a graph database, consisting of node data and edge data. Semantic class nodes are node data representing a specific data object category in the knowledge graph. Field names are the text identifiers for each column in the physical table, and field types are the column data type definitions. Attribute nodes are attribute node data representing the attributes possessed by semantic class nodes in the knowledge graph. Preset parent-class mapping rules store the correspondence between semantic roots and existing parent-class nodes in the knowledge graph. Inheritance edge data are directed edges in the graph data structure representing inheritance relationships between nodes. The first mapping edge connects semantic class nodes to physical table metadata objects, representing the correspondence between semantic class nodes and unmatched physical tables. The second mapping edge connects attribute nodes to physical table field metadata objects, representing the correspondence between attribute nodes and unmatched physical table fields.

[0009] The aforementioned technical solution forms a complete and closed-loop automated processing flow across five stages: mapping failure detection, semantic feature extraction, dynamic node creation, parent class inference and mounting, and mapping edge establishment. The technical benefits are as follows: First, it achieves full coverage of physical tables in the knowledge graph, increasing the knowledge graph's coverage of physical tables from less than 40% in traditional methods to nearly 100%, completely eliminating data gaps caused by discarding physical tables due to mapping failures. Second, through root word recognition and automatic parent class inference mechanisms, automatically generated semantic nodes possess basic semantic hierarchical relationships, exhibiting stronger semantic interpretability and systematic organization capabilities compared to simple direct mapping of physical table names. Third, through a two-layer mapping edge data structure based on unique metadata identifiers of physical tables, it achieves precise and automated conversion from semantic query intent to physical data access paths, significantly improving query response accuracy and automation.

[0010] In some technical solutions of this application, preprocessing, segmentation, and dictionary matching are performed on the physical table name string that does not match the physical table to determine at least one semantic root, including:

[0011] Perform preprocessing on the physical table name string to remove preset affixes to be removed that are stored in the computer. The preset affixes to be removed include at least one of the following: table prefix, system prefix, temporary table identifier, and year suffix.

[0012] Based on the preset segmentation rules stored in the computer, the preprocessed physical table name string is segmented to obtain at least one vocabulary unit. The preset segmentation rules include underscore segmentation rules and camelCase naming segmentation rules.

[0013] The vocabulary units are matched with a preset semantic dictionary stored in the computer. Successfully matched vocabulary units are mapped to corresponding semantic roots, or unmatched vocabulary units are used as semantic roots.

[0014] In some technical solutions of this application, a preset semantic dictionary stores the mapping relationship between code abbreviations and semantic roots, and maps successfully matched word units to corresponding semantic roots, including: mapping word units to corresponding standard semantic roots in the preset semantic dictionary.

[0015] In some technical solutions of this application, attribute nodes associated with semantic class nodes are created based on the field names and field types of unmatched physical tables, including:

[0016] Extract the field names and field types of all fields from the unmatched physical tables;

[0017] For each field, an attribute node is created based on the corresponding field name and field type, and attribute edge data is established between the semantic class node and the attribute node.

[0018] In some technical solutions of this application, both the first mapping edge data and the second mapping edge data contain unique metadata identifiers of the physical table.

[0019] In some technical solutions of this application, the existing parent nodes in the knowledge graph are determined based on the matching results of semantic roots and pre-stored preset parent class mapping rules, including:

[0020] Traverse the semantic roots and match them with the preset parent class mapping rules. The preset parent class mapping rules store the correspondence between semantic roots and parent class nodes.

[0021] If a matching parent node exists, the matched parent node will be determined as the parent node of the semantic class node.

[0022] If no parent node is matched for any semantic root, the preset top-level semantic node pre-stored in the computer is determined as the parent node of the semantic node.

[0023] In some technical solutions of this application, the first mapping edge data is used to represent the correspondence between semantic class nodes and unmatched physical tables, and the second mapping edge data is used to represent the correspondence between attribute nodes and fields of unmatched physical tables.

[0024] In some technical solutions of this application, the method for automatically generating physical table semantic nodes based on semantic analysis also includes:

[0025] Obtain the query request for the target database, which includes the target semantic identifier;

[0026] Retrieve semantic class nodes that match the target semantic identifier in the knowledge graph, including the semantic class nodes that were created;

[0027] Based on the first and second mapping edge data, determine the target physical table and target fields corresponding to the matched semantic class nodes;

[0028] Generate data query statements based on the target physical table and target fields.

[0029] A second aspect of the present invention provides an apparatus for automatically generating semantic nodes of physical tables based on semantic analysis, comprising:

[0030] The unmatched table acquisition module is used to acquire at least one unmatched physical table in the target database, as well as the unique metadata identifier of the physical table corresponding to the unmatched physical table. An unmatched physical table refers to a physical table that has no matching corresponding item after performing mapping matching on the physical tables in the target database based on predefined ontology mapping rules.

[0031] The semantic root determination module is used to perform preprocessing, segmentation, and dictionary matching on the physical table name string of unmatched physical tables to determine at least one semantic root.

[0032] The semantic node creation module is used to create semantic class nodes based on semantic roots within the knowledge graph stored in the graph database, and to create attribute nodes associated with the semantic class nodes based on the field names and field types of the unmatched physical tables.

[0033] The hierarchical relationship establishment module is used to determine the existing parent nodes in the knowledge graph based on the matching results of semantic roots and pre-stored preset parent class mapping rules, and to establish inheritance edge data between semantic nodes and parent nodes.

[0034] The mapping relationship establishment module is used to establish the first mapping edge data between semantic class nodes and unmatched physical tables in the knowledge graph based on the unique metadata identifier of the physical table, and to establish the second mapping edge data between attribute nodes and fields of unmatched physical tables.

[0035] A third aspect of the present invention provides an apparatus for automatically generating semantic nodes of physical tables based on semantic analysis, comprising: a processor and a memory, wherein the memory stores a program or instructions, and the processor, when executing the program or instructions in the memory, implements the steps of the method for automatically generating semantic nodes of physical tables based on semantic analysis as described in any of the above-described technical solutions. Therefore, the apparatus for automatically generating semantic nodes of physical tables based on semantic analysis possesses all the beneficial effects of the method for automatically generating semantic nodes of physical tables based on semantic analysis as described in any of the above-described technical solutions.

[0036] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0037] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0038] Figure 1 This is one of the flowcharts illustrating an automatic generation method for physical table semantic nodes based on semantic analysis according to an embodiment of the present invention.

[0039] Figure 2 This is a second flowchart illustrating an automatic generation method for physical table semantic nodes based on semantic analysis according to an embodiment of the present invention.

[0040] Figure 3 This is one of the schematic diagrams illustrating the verification case of automatic semantic node generation of physical table semantic nodes based on semantic analysis according to an embodiment of the present invention on two typical non-standard physical tables.

[0041] Figure 4 This is the second illustration of a verification case of the automatic generation of semantic nodes of physical tables based on semantic analysis according to an embodiment of the present invention, on two typical non-standard physical tables.

[0042] Figure 5 This is one of the schematic block diagrams of a physical table semantic node automatic generation device based on semantic analysis according to an embodiment of the present invention;

[0043] Figure 6 This is a second schematic block diagram of an automatic generation device for physical table semantic nodes based on semantic analysis according to an embodiment of the present invention. Detailed Implementation

[0044] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0045] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0046] The following reference Figures 1 to 6 This invention describes a method and apparatus for automatically generating semantic nodes of physical tables based on semantic analysis, according to some embodiments of the present invention.

[0047] like Figure 1 As shown, embodiments of this application provide a method for automatically generating semantic nodes of physical tables based on semantic analysis. The method is executed by a computer and includes the following steps:

[0048] Step 102: Obtain at least one unmatched physical table in the target database, and the unique metadata identifier of the physical table corresponding to the unmatched physical table. An unmatched physical table refers to a physical table that has no matching corresponding item after performing mapping matching on the physical tables in the target database based on predefined ontology mapping rules.

[0049] Step 104: Perform preprocessing, segmentation, and dictionary matching on the physical table name string that does not match the physical table to determine at least one semantic root word;

[0050] Step 106: Based on semantic roots, create semantic class nodes in the knowledge graph stored in the graph database, and create attribute nodes associated with semantic class nodes based on the field names and field types of the unmatched physical tables;

[0051] Step 108: Based on the matching results of semantic roots and pre-stored preset parent class mapping rules, determine the existing parent class nodes in the knowledge graph and establish inheritance edge data between semantic class nodes and parent class nodes.

[0052] Step 110: Based on the unique metadata identifier of the physical table, establish the first mapping edge data between semantic class nodes and unmatched physical tables in the knowledge graph, and establish the second mapping edge data between attribute nodes and fields of unmatched physical tables.

[0053] In this application, the target database is a database system instance storing physical tables to be processed. A physical table is the basic unit of data actually stored in the target database. Predefined ontology mapping rules are a set of mapping logic pre-configured in the computer system, used to establish a correspondence between physical table names and field names and concepts and attributes in the standard ontology library. An unmatched physical table refers to a physical table in the target database that has no matching entries after mapping matching based on the predefined ontology mapping rules. A unique metadata identifier for a physical table is an identifier used within the database system to uniquely identify a physical table, providing the foundation for establishing precise mapping edge data between semantic nodes and physical tables. The physical table name string is an identifier text defined by the developers when the database table is created. Preprocessing operations remove preset affixes to be removed from the physical table name string. Segmentation operations divide the preprocessed string according to preset segmentation rules to obtain multiple independent lexical units. Dictionary matching processing compares the segmented lexical units with a preset semantic dictionary to obtain semantic roots. Semantic roots are semantic lexical units that represent the data object categories in the physical table name string. Graph databases are database systems that use graph data structures as their core storage and query model. Knowledge graphs are the overall graph data structure stored within a graph database, consisting of node data and edge data. Semantic class nodes are node data representing a specific data object category in the knowledge graph. Field names are the text identifiers for each column in the physical table, and field types are the column data type definitions. Attribute nodes are attribute node data representing the attributes possessed by semantic class nodes in the knowledge graph. Preset parent-class mapping rules store the correspondence between semantic roots and existing parent-class nodes in the knowledge graph. Inheritance edge data are directed edges in the graph data structure representing inheritance relationships between nodes. The first mapping edge connects semantic class nodes to physical table metadata objects, representing the correspondence between semantic class nodes and unmatched physical tables. The second mapping edge connects attribute nodes to physical table field metadata objects, representing the correspondence between attribute nodes and unmatched physical table fields.

[0054] The aforementioned technical solution forms a complete and closed-loop automated processing flow across five stages: mapping failure detection, semantic feature extraction, dynamic node creation, parent class inference and mounting, and mapping edge establishment. The technical benefits are as follows: First, it achieves full coverage of physical tables in the knowledge graph, increasing the knowledge graph's coverage of physical tables from less than 40% in traditional methods to nearly 100%, completely eliminating data gaps caused by discarding physical tables due to mapping failures. Second, through root word recognition and automatic parent class inference mechanisms, automatically generated semantic nodes possess basic semantic hierarchical relationships, exhibiting stronger semantic interpretability and systematic organization capabilities compared to simple direct mapping of physical table names. Third, through a two-layer mapping edge data structure based on unique metadata identifiers of physical tables, it achieves precise and automated conversion from semantic query intent to physical data access paths, significantly improving query response accuracy and automation.

[0055] In some embodiments of this application, preprocessing, segmentation, and dictionary matching are performed on the physical table name string that does not match the physical table to determine at least one semantic root. This includes: performing preprocessing on the physical table name string to remove preset affixes to be removed that are pre-stored in the computer, the preset affixes to be removed including at least one of table prefix, system prefix, temporary table identifier, and year suffix; performing word segmentation on the preprocessed physical table name string according to preset segmentation rules pre-stored in the computer to obtain at least one lexical unit, the preset segmentation rules including underscore segmentation rules and camelCase naming segmentation rules; matching the lexical unit with a preset semantic dictionary pre-stored in the computer, mapping the successfully matched lexical unit to the corresponding semantic root, or using the unmatched lexical unit as the semantic root.

[0056] In the above embodiments, the preset affixes to be removed are a set of character fragments pre-configured in the computer system, including table prefixes such as t_, system prefixes such as sys_, temporary table identifiers such as tmp_, and year suffixes such as _2023. The preprocessing operation uses string matching to identify and delete these affixes. The preset segmentation rules include underscore segmentation rules and camelCase naming segmentation rules. The underscore segmentation rule uses underscore characters as delimiters to divide the string into multiple substrings, while the camelCase naming segmentation rule uses uppercase letters as the basis for word boundary identification. A preset semantic dictionary stores the mapping relationship between code abbreviations and semantic roots. These technical features, by clearly defining the preset affix set and preset segmentation rules, transform the process of extracting semantic features from physical table name strings into a character processing algorithm that can be precisely executed by a computer, improving the automation level and processing consistency of semantic feature extraction.

[0057] Taking the physical table name t_proj_mgr_ext_2023 as an example, the computer first performs preprocessing to remove the t_ prefix and the _2023 year suffix to obtain proj_mgr_ext. Then, it is segmented according to the underscore splitting rule to obtain three lexical units: proj, mgr, and ext. Next, the lexical units are matched with a preset semantic dictionary. proj is mapped to Project, mgr is mapped to Manager, and ext is retained as semantic roots. Finally, the semantic root set is determined to be Project, Manager, and Extension.

[0058] Specifically, the system includes a pre-defined template for matching affixes to be removed based on regular expressions, pre-defined parameters for the priority order of segmentation rules, a pre-defined semantic dictionary hierarchical matching strategy, and a manual review trigger mechanism when matching fails.

[0059] In some embodiments of this application, a preset semantic dictionary stores a mapping relationship between code abbreviations and semantic roots, and maps successfully matched word units to corresponding semantic roots, including: mapping word units to corresponding standard semantic roots in the preset semantic dictionary.

[0060] In the above embodiments, code abbreviations are shorthand forms used by software developers when naming database tables, such as Mgr, Cfg, Dtl, etc. Standard semantic roots are standard terms that describe complete business semantic concepts. A pre-defined semantic dictionary stores mapping relationships in a key-value pair data structure. Through the standardized mapping mechanism of the pre-defined semantic dictionary, heterogeneous physical table names from different business systems are uniformly mapped to consistent standardized semantic roots, ensuring consistency in naming of subsequently created semantic class nodes. This avoids the redundant problem of repeatedly creating isolated nodes in the knowledge graph due to naming differences, improving the semantic consistency and data quality of the knowledge graph.

[0061] Taking three physical tables t_user_mgr, sys_user_manager, and user_manage_t as examples, after preprocessing, segmentation, and dictionary matching, the same standard semantic roots User and Manager are obtained. Based on the same standard semantic roots, the computer creates a unified semantic class node in the knowledge graph, realizing a unified semantic representation of heterogeneous physical tables.

[0062] Specifically, the pre-defined semantic dictionary hierarchical structure design includes a two-level matching architecture of a general technical dictionary layer and an industry terminology dictionary layer, an edit distance-based fuzzy matching fault tolerance mechanism, an automatic expansion and learning mechanism for the pre-defined semantic dictionary, and a standard semantic root version management mechanism.

[0063] In some embodiments of this application, attribute nodes associated with semantic class nodes are created based on the field names and field types of unmatched physical tables, including:

[0064] Extract the field names and field types of all fields from the unmatched physical tables;

[0065] For each field, an attribute node is created based on the corresponding field name and field type, and attribute edge data is established between the semantic class node and the attribute node.

[0066] In the above embodiments, the field name is the text identifier of each column in the relational database table, and the field type is the column data type definition, such as integer, string, date / time, etc. Attribute nodes are stored as node data in the knowledge graph, containing metadata information such as attribute node identifier, attribute name, and attribute data type. Attribute edge data is the directed edge in the graph data structure representing the affiliation relationship between semantic class nodes and attribute nodes. These technical features achieve automated granular mapping from physical table fields to semantic attribute nodes, fully modeling field-level metadata in the knowledge graph, making the fields themselves independently searchable graph node objects. The field type information recorded in the attribute nodes plays a crucial role in the subsequent query statement generation stage, allowing the computer to automatically adapt the value format in the query conditions based on the field type.

[0067] Taking the unmatched physical table t_proj_mgr_ext, which contains the fields proj_id (INT), proj_name (VARCHAR), mgr_name (VARCHAR), and ext_data (TEXT), as an example, the computer creates an attribute node for each field and establishes a connection with the semantic class node ProjMgrExt through attribute edge data. When generating the query statement, the computer recognizes that the mgr_name attribute node is of string type and automatically adds quotation marks to the values ​​in the query conditions to generate the correct SQL predicate conditions.

[0068] Specifically, the rules table for mapping and converting field types to semantic data types, the rules for generating attribute node identifiers, the additional metadata in attribute edge data records such as whether a field is a primary key identifier and whether null values ​​are allowed, and the aggregation modeling method for composite attribute nodes.

[0069] In some embodiments of this application, both the first mapping edge data and the second mapping edge data contain a unique metadata identifier for the physical table.

[0070] In the above embodiments, the unique metadata identifier of a physical table is an identifier used within the database system to uniquely identify a physical table. It can be an internal object identifier assigned by the database system or a fully qualified identifier string formed by combining the data source identifier, database name, schema name, and table name. When the first mapping edge data is established, the computer writes the unique metadata identifier of the physical table as edge attribute data; similarly, when the second mapping edge data is established, the computer writes the unique metadata identifier of the physical table into the edge attribute field. These technical features enable precise binding between semantic layer nodes and physical layer storage objects based on system-level unique identifiers, ensuring the stability and persistence of the mapping relationship. Even when a physical table undergoes a renaming operation, as long as the unique metadata identifier of the physical table remains unchanged, the mapping edge data established based on the unique metadata identifier will remain valid, avoiding the breakage of semantic links due to table name changes. In infringement comparison scenarios, the unique metadata identifier of the physical table, as an objectively existing technical feature, can be directly verified, reducing the technical threshold and difficulty of evidence collection for rights protection.

[0071] Taking the migration of the physical table t_proj_mgr_dev from the development environment to the production environment and renaming it to t_proj_mgr_prod as an example, the unique metadata identifiers of the two tables are DEV_DB.dbo.t_proj_mgr_dev and PROD_DB.dbo.t_proj_mgr_prod, respectively. The computer creates semantic class nodes for the two physical tables and establishes mapping edge data. The mapping edge data records the unique metadata identifiers of each physical table, thereby achieving data isolation across multiple environments and accurate maintenance of mapping relationships.

[0072] Specifically, the unique metadata identifier of the physical table is defined by its specific structure, the additional metadata fields recorded in the mapping edge data are such as the mapping creation timestamp and mapping confidence score, and the mechanism for associating the unique metadata identifier of the physical table with the historical version identifier for tracing.

[0073] In some embodiments of this application, the existing parent nodes in the knowledge graph are determined based on the matching results of semantic roots and pre-stored preset parent class mapping rules, including:

[0074] Traverse the semantic roots and match them with the preset parent class mapping rules. The preset parent class mapping rules store the correspondence between semantic roots and parent class nodes.

[0075] If a matching parent node exists, the matched parent node will be determined as the parent node of the semantic class node.

[0076] If no parent node is matched for any semantic root, the preset top-level semantic node pre-stored in the computer is determined as the parent node of the semantic node.

[0077] In the above embodiments, the preset parent class mapping rule can be implemented in the form of a hash mapping table, where the key is a semantic root string and the value is the identifier of the corresponding parent class node. The preset top-level semantic node is the root node of the semantic hierarchy in the knowledge graph. The effect of the above-mentioned hierarchical determination logic is to automatically assign a reasonable hierarchical affiliation to the automatically generated semantic class nodes, so that the semantic class nodes are attached to the category parent node or the top-level root node, ensuring the integrity of the knowledge graph hierarchy and the closed-loop processing of node attachment, and avoiding the generation of isolated nodes.

[0078] Taking the physical table name sys_log_2023 as an example, after processing to obtain the semantic roots Sys and Log, the Log root hits the parent node of the log class, thus attaching the semantic node under the parent node of the log class. In another embodiment, the semantic roots temp_custom_report do not hit the parent node, and the semantic node is attached to the preset top-level semantic node.

[0079] Specifically, the priority determination rules for multi-semantic root matching of different parent nodes, the incremental update mechanism of preset parent mapping rules, and the inheritance source identifier field recorded in the inheritance edge data.

[0080] In some embodiments of this application, the first mapping edge data is used to characterize the correspondence between semantic class nodes and unmatched physical tables, and the second mapping edge data is used to characterize the correspondence between attribute nodes and fields of unmatched physical tables.

[0081] In the above embodiments, two mapping edge data constitute a two-layer mapping link from the semantic layer to the physical storage layer. The first mapping edge is used for table-level positioning, and the second mapping edge is used for field-level positioning. The technical advantage lies in providing a complete graph traversal path for semantic queries. During a query, the computer first locates the target physical table along the first mapping edge, and then locates the target field along the second mapping edge, automatically generating a data query statement accordingly. The entire process achieves accurate transformation based on explicit edge data, eliminating the need for manually written mapping rules. The mapping edge data can be directly observed and verified during infringement comparisons, reducing the difficulty of evidence collection.

[0082] Taking querying extended information of a project manager as an example, the computer locates the t_proj_mgr_ext table along the first mapping edge, and locates the proj_id, mgr_name, and ext_data fields along the second mapping edge, generating the SQL query statement SELECT proj_id, mgr_name, ext_data FROM t_proj_mgr_ext.

[0083] Specifically, the mapped edge data records the mapping confidence score, the mapping relationship validity verification mechanism, and the specific field definitions of edge attributes.

[0084] In some embodiments of this application, a query request for a target database is obtained, the query request including a target semantic identifier; semantic class nodes matching the target semantic identifier are retrieved in the knowledge graph, including the created semantic class nodes; based on the first mapping edge data and the second mapping edge data, the target physical table and target field corresponding to the matched semantic class node are determined; and a data query statement is generated based on the target physical table and target field.

[0085] In the above embodiments, the query request is a data retrieval request initiated by the user or upper-layer application targeting the database. The target semantic identifier is a keyword or identifier carried in the query request to represent the target concept. The data query statement is an executable query command generated according to the target database type and syntax specifications. These technical features automatically generate semantic nodes and mapping edge data and apply them to end-to-end semantic query scenarios, enabling previously unmatched physical tables that could not be covered by semantic retrieval to respond to query requests. Experiments show that after applying this method, the system went from returning "no relevant table found" to successfully locating dynamically generated semantic nodes and generating correct SQL statements, increasing query coverage from approximately 35% to nearly 100%.

[0086] Taking querying the system logs for the most recent week as an example, the computer locates the SysLog semantic node and the sys_log_2023 physical table, and generates a data query statement with time conditions: SELECT log_content FROM sys_log_2023 WHERE create_time >= DATE_SUB(NOW(), INTERVAL 7 DAY).

[0087] Specifically, it includes field type adaptation handling during query statement generation, SQL injection protection escaping mechanism, and automatic derivation logic for multi-table join conditions.

[0088] like Figure 2 As shown, embodiments of this application provide a method for automatically generating semantic nodes of physical tables based on semantic analysis, the steps of which include:

[0089] Step 201: Perform noise reduction preprocessing on the physical table name strings that do not match the physical table, remove the preset prefixes to be removed, and obtain the noise-reduced table name strings;

[0090] Step 202: According to the preset segmentation rules, perform word segmentation on the denoised table name string to obtain multiple word units;

[0091] Step 203: Perform root word recognition processing on all word units and match each word unit with a preset semantic dictionary;

[0092] Step 204: Determine whether the word unit is successfully matched. If the determination is yes, proceed to step 205; if the determination is no, proceed to step 206.

[0093] Step 205: Map the successfully matched lexical units to the corresponding standard semantic roots, and proceed to step 207;

[0094] Step 206: Directly use the word units that failed to match as semantic roots and proceed to step 207;

[0095] Step 207: Summarize all semantic roots corresponding to the unmatched physical tables;

[0096] Step 208: Construct semantic class node names based on all semantic roots;

[0097] Step 209: Perform parent class inference processing, matching each semantic root with the pre-stored preset parent class mapping rules;

[0098] Step 210: Determine if there is a matching parent node. If yes, proceed to step 211; if no, proceed to step 212.

[0099] Step 211: Determine the parent node of the matched node as the parent node of the semantic node, and proceed to step 213.

[0100] Step 212: Determine the preset top-level semantic node as the parent node of the semantic class node, and proceed to step 213;

[0101] Step 213: Create a dynamic semantic class node that uniquely corresponds to the unmatched physical table in the knowledge graph stored in the graph database;

[0102] Step 214: Establish inheritance edge data between semantic class nodes and parent class nodes;

[0103] Step 215: Based on the field names and field types of the unmatched physical tables, create attribute nodes associated with the semantic class nodes;

[0104] Step 216: Establish attribute edge data between semantic class nodes and attribute nodes;

[0105] Step 217: Obtain the unique metadata identifier of the physical table corresponding to the unmatched physical table;

[0106] Step 218: Based on the unique metadata identifier of the physical table, establish the first mapping edge data in the knowledge graph. The first mapping edge data is used to represent the correspondence between semantic class nodes and unmatched physical tables.

[0107] Step 219: Establish second mapping edge data, which is used to represent the correspondence between attribute nodes and corresponding fields of unmatched physical tables;

[0108] Step 220: Complete the entire process of generating dynamic semantic nodes corresponding to unmatched physical tables.

[0109] like Figure 3 and Figure 4 As shown, Figure 3 and Figure 4 The document presents verification cases of automatic semantic node generation for two typical non-standard physical tables. Case 1, targeting the business extension table t_proj_mgr_ext, first cleanses the table name to obtain ProjMgrExt, then segments it into two lexical units, Proj and Mgr. After matching the root word library, the root word Proj is found, confirming its semantics as project-related. Finally, a semantic node ent:ProjMgrExt is generated and automatically inherits from the parent class ent:ProjectObject. The verification conclusion is that the semantic capture is accurate. Case 2, targeting the system log table sys_log_2023, first cleanses the table name to obtain SysLog, then segments it into two lexical units, Sys and Log. After matching the root word library, the root word Log is found, confirming its semantics as log recording. Finally, a semantic node ent:SysLog is generated and automatically inherits from the parent class ent:Log. The verification conclusion is that the classification hierarchy is correct. Both cases jointly verify the semantic extraction and node generation capabilities of this method for different types of non-standard physical tables.

[0110] like Figure 5As shown, embodiments of this application provide an automatic generation device 300 for physical table semantic nodes based on semantic analysis, including: an unmatched table acquisition module 310, a semantic root determination module 320, a semantic node creation module 330, a hierarchical relationship establishment module 340, and a mapping relationship establishment module 350. The unmatched table acquisition module 310 is used to acquire at least one unmatched physical table in the target database, and the unique metadata identifier of the physical table corresponding to the unmatched physical table. An unmatched physical table refers to a physical table that has no matching corresponding item after performing mapping matching on the physical tables in the target database based on predefined ontology mapping rules. The semantic root determination module 320 is used to perform preprocessing, segmentation, and dictionary matching processing on the physical table name string of the unmatched physical table. The system identifies at least one semantic root; a semantic node creation module 330 is used to create semantic class nodes in the knowledge graph stored in the graph database based on the semantic root, and to create attribute nodes associated with the semantic class nodes based on the field names and field types of the unmatched physical tables; a hierarchical relationship establishment module 340 is used to determine the existing parent nodes in the knowledge graph based on the matching results of the semantic root and the pre-stored preset parent class mapping rules, and to establish inheritance edge data between the semantic class nodes and the parent nodes; a mapping relationship establishment module 350 is used to establish the first mapping edge data between the semantic class nodes and the unmatched physical tables in the knowledge graph based on the unique metadata identifier of the physical tables, and to establish the second mapping edge data between the attribute nodes and the fields of the unmatched physical tables.

[0111] This application provides a semantic analysis-based automatic generation device 300 for physical table semantic nodes. Through modular collaborative design, it achieves fully automated generation of semantic nodes for non-standard physical tables. This precisely solves the problems of low knowledge graph coverage and data black holes caused by discarding or storing long-tail physical tables that cannot be mapped to standard ontologies as raw data, as seen in existing technologies. The unmatched table acquisition module 310 accurately identifies physical tables without corresponding ontology mapping rules and obtains their unique metadata identifiers. The semantic root determination module 320 completes denoising, word segmentation, and standardized root extraction of the table name string. The semantic node creation module... The module 330 automatically generates corresponding semantic class nodes and attribute nodes. The hierarchical relationship establishment module 340 realizes automatic inference of parent classes and semantic hierarchical mounting. The mapping relationship establishment module 350 establishes a two-level precise mapping at the table level and field level based on unique metadata identifiers. Overall, it increases the coverage of the knowledge graph to the physical tables of the database from the traditional 35% to nearly 100%. At the same time, it gives the automatically generated nodes basic classification semantics and interpretability. It can complete the semantic processing of non-standard tables in heterogeneous systems without manual intervention, which significantly improves the efficiency of enterprise-wide data asset governance and the response accuracy of semantic query system.

[0112] like Figure 6As shown, an embodiment of this application provides an automatic physical table semantic node generation device 400 based on semantic analysis, including a processor 402 and a memory 404. The memory 404 stores a program or instructions. When the processor 402 executes the program or instructions in the memory 404, it implements the steps of the automatic physical table semantic node generation method based on semantic analysis as described in any of the above embodiments. Therefore, the automatic physical table semantic node generation device 400 based on semantic analysis possesses all the beneficial effects of the automatic physical table semantic node generation method based on semantic analysis as described in any of the above embodiments.

[0113] Embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the semantic analysis-based automatic generation method for physical table semantic nodes as described in any of the above embodiments. Therefore, the readable storage medium possesses all the beneficial effects of the semantic analysis-based automatic generation method for physical table semantic nodes as described in any of the above embodiments.

[0114] In the claims, description, and accompanying drawings of this invention, the term "plural" refers to two or more. Unless otherwise explicitly defined, the terms "upper," "lower," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and simplifying the descriptive process, and are not intended to indicate or imply that the device or element referred to must have the described specific orientation, or be constructed and operated in a specific orientation. Therefore, these descriptions should not be construed as limiting the invention. The terms "connected," "installed," "fixed," etc., should be interpreted broadly. For example, "connected" can be a fixed connection between multiple objects, a detachable connection between multiple objects, or an integral connection; it can be a direct connection between multiple objects or an indirect connection between multiple objects through an intermediate medium. For those skilled in the art, the specific meaning of the above terms in this invention can be understood based on the specific circumstances described above.

[0115] In the claims, description, and accompanying drawings of this invention, the terms "one embodiment," "some embodiments," "specific embodiment," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the invention. In the claims, description, and accompanying drawings of this invention, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0116] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for automatically generating semantic nodes of physical tables based on semantic analysis, characterized in that, The method is executed by a computer and includes: Obtain at least one unmatched physical table in the target database, and the unique metadata identifier of the physical table corresponding to the unmatched physical table. The unmatched physical table refers to a physical table that has no matching corresponding item after performing mapping matching on the physical tables in the target database based on predefined ontology mapping rules. Perform preprocessing, segmentation, and dictionary matching on the physical table name string of the unmatched physical table to determine at least one semantic root word; Based on the semantic roots, semantic class nodes are created in the knowledge graph stored in the graph database, and attribute nodes associated with the semantic class nodes are created based on the field names and field types of the unmatched physical tables. Based on the matching results between the semantic root and the pre-stored preset parent class mapping rules, the existing parent class nodes in the knowledge graph are determined, and the inheritance edge data between the semantic class node and the parent class node is established. Based on the unique metadata identifier of the physical table, a first mapping edge data is established between the semantic class node and the unmatched physical table in the knowledge graph, and a second mapping edge data is established between the attribute node and the field of the unmatched physical table.

2. The method according to claim 1, characterized in that, The preprocessing, segmentation, and dictionary matching of the physical table name string of the unmatched physical table to determine at least one semantic root word includes: The physical table name string is preprocessed to remove preset affixes to be removed that are stored in the computer. The preset affixes to be removed include at least one of table prefix, system prefix, temporary table identifier, and year suffix. According to the preset segmentation rules stored in the computer, the preprocessed physical table name string is segmented to obtain at least one vocabulary unit. The preset segmentation rules include underscore segmentation rules and camelCase naming segmentation rules. The vocabulary units are matched with a preset semantic dictionary stored in the computer. Successfully matched vocabulary units are mapped to corresponding semantic roots, or unmatched vocabulary units are used as semantic roots.

3. The method according to claim 2, characterized in that, The preset semantic dictionary stores the mapping relationship between code abbreviations and semantic roots. Mapping the successfully matched word units to the corresponding semantic roots includes: mapping the word units to the corresponding standard semantic roots in the preset semantic dictionary.

4. The method according to claim 1, characterized in that, The step of creating an attribute node associated with the semantic class node based on the field names and field types of the unmatched physical table includes: Extract the field names and field types of all fields from the unmatched physical table; For each field, an attribute node is created based on the corresponding field name and field type, and attribute edge data is established between the semantic class node and the attribute node.

5. The method according to claim 1, characterized in that, Both the first mapping edge data and the second mapping edge data contain the unique metadata identifier of the physical table.

6. The method according to claim 1, characterized in that, The step of determining the existing parent nodes in the knowledge graph based on the matching results of the semantic roots and the pre-stored preset parent class mapping rules includes: Traverse the semantic roots and match them with the preset parent class mapping rules. The preset parent class mapping rules store the correspondence between semantic roots and parent class nodes. If a matching parent node exists, the matched parent node will be determined as the parent node of the semantic class node. If no parent node is matched for any of the semantic roots, the preset top-level semantic node pre-stored in the computer is determined as the parent node of the semantic node.

7. The method according to claim 1, characterized in that, The first mapping edge data is used to characterize the correspondence between the semantic class node and the unmatched physical table, and the second mapping edge data is used to characterize the correspondence between the attribute node and the field of the unmatched physical table.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Obtain a query request for the target database, wherein the query request includes a target semantic identifier; Retrieve semantic class nodes that match the target semantic identifier in the knowledge graph, including the created semantic class nodes; Based on the first mapping edge data and the second mapping edge data, determine the target physical table and target field corresponding to the matched semantic class node; Generate a data query statement based on the target physical table and the target field.

9. A device for automatically generating semantic nodes of physical tables based on semantic analysis, characterized in that, include: The unmatched table acquisition module is used to acquire at least one unmatched physical table in the target database, and the unique metadata identifier of the physical table corresponding to the unmatched physical table. The unmatched physical table refers to a physical table that has no matching corresponding item after performing mapping matching on the physical tables in the target database based on predefined ontology mapping rules. The semantic root determination module is used to perform preprocessing, segmentation, and dictionary matching on the physical table name string of the unmatched physical table to determine at least one semantic root; The semantic node creation module is used to create semantic class nodes in the knowledge graph stored in the graph database based on the semantic root words, and to create attribute nodes associated with the semantic class nodes based on the field names and field types of the unmatched physical tables. The hierarchical relationship establishment module is used to determine the existing parent nodes in the knowledge graph based on the matching results of the semantic root and the pre-stored preset parent class mapping rules, and to establish inheritance edge data between the semantic node and the parent node. The mapping relationship establishment module is used to establish first mapping edge data between the semantic class node and the unmatched physical table in the knowledge graph based on the unique metadata identifier of the physical table, and to establish second mapping edge data between the attribute node and the field of the unmatched physical table.

10. A device for automatically generating semantic nodes of physical tables based on semantic analysis, characterized in that, include: processor; A memory storing a program or instructions, wherein the processor, when executing the program or instructions, implements the steps of the method as described in any one of claims 1 to 8.

Images