Unified orchestration and query execution for data access, computer equipment and storage media

Abstract

This invention discloses a unified data access orchestration and query execution method, a computer device, and a storage medium, belonging to the field of computer technology. The unified data access orchestration and query execution method includes the following steps: S1, determining query parameters; S2, constructing a unified query model; S3, identifying field attribution and modeling query paths; and S4, reasoning and selecting expected paths. This invention achieves a unified structured expression of complex query logic, supports automatic identification of field attribution and path reasoning, and can automatically translate query statements and distribute them to the corresponding databases for execution. It provides an intermediate layer integration mechanism to achieve the final structured return of data. This invention can effectively reduce the complexity of access and management between multiple database systems, improve the flexibility, scalability, and development efficiency of data queries, and is applicable to various scenarios such as data platforms, reporting systems, and data service API platforms.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and in particular relates to a unified orchestration and query execution method for cross-database platform data access, computer equipment, and storage medium. Background Technology

[0002] In recent years, with the development of big data, cloud computing, and search engine technologies, database technology has shown a trend of diversification and rapid iteration. Traditional relational databases such as MySQL and PostgreSQL are continuously optimizing their performance and high-availability architectures, while columnar databases such as ClickHouse and Doris are widely used due to their high performance in massive data analysis scenarios. At the same time, databases for search and semi-structured data, such as Elasticsearch and OpenSearch, are gradually becoming important components of enterprise data processing. The pace of database system updates and replacements continues to accelerate, with new technologies emerging one after another, making enterprises face greater uncertainty in database technology selection and architecture evolution.

[0003] In practical applications, different business systems often rely on different types of databases to meet their varying needs in terms of performance, storage structure, and query characteristics. This leads to a situation where an enterprise may deploy multiple database systems simultaneously, requiring the development of dedicated data access logic for each database. More seriously, if an enterprise needs to switch its underlying database due to factors such as performance, cost, security, or cloud-native requirements—for example, from MySQL to ClickHouse or Elasticsearch—it often requires a large-scale refactoring of the original query interfaces, service logic, and front-end integration processes. This migration is costly, time-consuming, and can even impact business continuity.

[0004] While some database middleware, ORM frameworks, and query builder tools have emerged to alleviate this problem, most solutions only support specific database types and lack a truly universal, unified query encapsulation capability applicable to multiple database systems. Implementation details such as query syntax, data type mapping, pagination strategies, and sorting methods still require application developers to handle manually, resulting in poor system scalability, high development costs, and heavy maintenance workload.

[0005] In summary, existing technologies face the following prominent problems in the context of multiple database systems coexisting and the rapid evolution of database technology:

[0006] 1. The database syntax and query model differ greatly, lacking a unified abstraction layer.

[0007] Different database systems use different query languages ​​such as SQL and DSL, which have semantic and syntactic differences in field expression, filtering logic, aggregation methods, sorting and pagination. Existing systems often need to implement query logic separately for each database, lacking unified parameter specifications and query models, which makes development difficult and maintenance costly.

[0008] 2. Database switching is costly and the system has insufficient scalability.

[0009] Because the query logic is highly coupled with the specific database implementation, if an enterprise needs to switch database systems for purposes such as performance optimization, cost control, or technology upgrades, it often needs to rewrite a large amount of business logic and interface code, and even adjust the front-end interaction method. The migration cost is high, the cycle is long, and the flexibility and scalability of the system are severely restricted.

[0010] 3. The lack of a unified mechanism for result encapsulation and data structure increases the burden of front-end and back-end adaptation.

[0011] The data formats returned by different databases vary significantly. Some systems, such as Elasticsearch, return nested JSON, while relational databases mainly use table structures. Existing systems usually need to manually convert the format at the application layer, which leads to complex front-end calls, high code duplication, and is not conducive to the formation of a standardized data access protocol.

[0012] 4. Difficult to support heterogeneous concurrent access scenarios of databases.

[0013] In a multi-database deployment environment, if it is necessary to simultaneously access ClickHouse for analysis, access MySQL to retrieve master data, and access Elasticsearch to retrieve full text, existing technologies cannot provide unified encapsulation and concurrency scheduling capabilities. Developers need to call multiple interfaces and handle different response logics separately, which seriously affects system integration efficiency.

[0014] 5. Poor compatibility with new databases or cloud databases hinders technological evolution.

[0015] With the rise of new types of databases such as cloud-native databases, graph databases, and time-series databases, existing query systems often need to be customized and adapted, and cannot quickly access new data sources, which restricts the speed of technical architecture evolution and the digital transformation capabilities of enterprises.

[0016] In summary, existing technologies urgently need a general query encapsulation method and system that can effectively shield the differences in underlying databases, quickly adapt to multiple database systems, achieve unified query encapsulation, semantic adaptation, standardized result format, good scalability, and seamless query switching, so as to reduce development complexity, improve system versatility and future maintainability.

[0017] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0018] The purpose of this invention is to solve the above-mentioned technical problems and provide a unified orchestration and query execution method for cross-database platform data access, computer equipment and storage medium.

[0019] The first aspect of this invention provides a unified data access orchestration and query execution method, comprising the following steps: S1. Determine the query parameters, that is, determine the query parameters based on the query request provided by the user. The query request must provide the structured query parameters. S2. Construction of a unified query model, which involves parsing the query parameters into a unified query model for use across databases; S3. Field attribution identification and query path modeling, namely, constructing a graph structure relationship model between the fields corresponding to the query parameters and the backup table, wherein the backup table includes at least one of the following data entities in the database: table, set; S4. Expected path reasoning and selection, that is, enumerating all query paths and selecting the optimal query path based on the strategy.

[0020] In one embodiment of the present invention, the query parameters include: primary key field filtering parameters, fuzzy matching parameters, range parameters, boolean state parameters, pagination and sorting parameters, query field parameters, count control parameters, advanced custom query parameters, and array operation parameters.

[0021] In one embodiment of the present invention, step S2 includes: S21. Parameter filtering item extraction: Extract all query parameters used to query the true intent of the query request, i.e., the actual query parameters, into filtering conditions, or simply conditions. S22, Field semantic recognition: Based on the field corresponding to each actual query parameter, find its registration information in the field graph, and determine the database, spare table or document to which it belongs. The field graph refers to a pre-established field meta-information registration structure used to record the semantic information, structural position and database affiliation of all queryable fields. S23. Semantic type classification: Divide each of the conditions into different types, and the different types include at least one of the following types: exact matching, fuzzy matching, range matching, logical combination, and sorting operation; S24. Construction of a unified semantic structure: All the conditions are uniformly converted into a set of intermediate format data bodies, wherein the intermediate format data body represents a composite query condition, and the relationship between the conditions is AND; S26. Logical verification and normalization: Verify the legality of the intermediate format data body and transform the intermediate format data body into an intermediate abstract query model; S27. Syntax Abstraction Representation Generation: Based on the target database type, the intermediate format data body is mapped into a unified abstract syntax tree or semantic representation tree, which serves as the input for subsequent queries.

[0022] In one embodiment of the present invention, step S2 further includes the step: S25. Custom Logic Tree Fusion: If the query parameter is sql_custom_query_list and it is not empty, merge it with the intermediate format data body to construct a final logic query tree.

[0023] In one embodiment of the present invention, step S3 includes: S31, Field attribution resolution, including: First, a global field dictionary is constructed and maintained. This global field dictionary records all fields involved in the query and their metadata, including: The field name refers to each query parameter, the data source type, the alternate table name, the field type and data type, whether it is a primary key, foreign key or index field, and the relationship with the primary keys of other databases; Secondly, when a user submits a set of structured query parameters, each field corresponding to the set of structured query parameters is processed as follows: Field normalization and field attribution; S32. Graph structure modeling, including: A database graph is constructed and maintained through the foreign key relationships between the standby tables. This involves modeling the relationships between all the standby tables as a directed graph, i.e., the database graph. The directed graph includes graph nodes and graph edges. The graph nodes represent all the required data entities in the database, i.e., the standby tables, including: tables in relational databases, tables in columnar databases, collections in document-oriented databases, data tables in time-series databases, and entity nodes in graph databases. There is a graph edge between every two graph nodes that have a field relationship. The graph edge reflects the field relationship between the two graph nodes. The field relationship includes: foreign key relationship and edge attributes. The edge attributes include the associated field name, connection type (1:1, 1:N), connection direction, and field attributes of the two graph nodes connected by any graph edge. The associated field name indicates which two fields are associated. The connection type includes 1:1 and 1:N. The 1:1 connection type means that one graph node directly connects to only one other graph node. The 1:N connection type means that one graph node directly connects to N other graph nodes, where N is an integer greater than 1. The connection direction indicates which field of the two graph nodes points to the corresponding field of the other graph node. The field attributes include: the data type and constraints of the field. S33, Graph Path Reasoning and Query Relationship Planning.

[0024] In one embodiment of the present invention, step S33 includes: When the directed graph is used to process multi-field queries, the method for reasoning connection paths in the directed graph includes: S331. Assign and locate the multiple fields to obtain their respective graph nodes; S332. Calculate the different connection paths between the graph nodes obtained in step S331 in the directed graph to determine the optimal access method between different data entities; S333. For each of the multiple fields, generate a directed acyclic graph of subquery paths to determine the dependency order of data access at each step.

[0025] In one embodiment of the present invention, step S4 includes: S41. Global data relationship graph modeling and entity relationship abstraction: This involves maintaining a global data relationship graph based on the database graph. The global data relationship graph corresponds to the data relationship graph between entities in the query object. The global data relationship graph includes multiple nodes and multiple edges. Each node corresponds one-to-one with a graph node in the database graph, representing a spare table in a query object. Each edge represents a logical relationship between different entities. The logical relationship includes at least one of the following relationships: membership, classification, association, and upstream / downstream dependency. Each edge has a direction attribute: directed or undirected, and includes multiple semantic labels, time labels, source information, and credibility scores. Each edge in the multiple edges has a one-to-one corresponding graph edge in the database graph. S42. Multidimensional path weight modeling and quantitative evaluation remember For any given path between any two specified nodes in the global data relationship graph, its path score is calculated. The calculation formula is as follows: , In the above formula, Represents the first connection in the global data relationship graph. i Nodes and the j Nodes edge The corrected weights, where, i and j All are integers; For path The path structure adjustment function, Remember the edge The basic weight is , The weights are adjusted based on time decay, credibility, and source level. The expression is as follows: , For node pairs and The type adjustment function is used to reflect the edge The importance level of the connected nodes is called the node level. The node entity corresponding to the node level, that is, the entity included or associated with the fields in the node, is a high-level entity, a medium-level entity, or a low-level entity. The high-level entities include enterprises and patents; the medium-level entities include industrial chains, industrial links, technologies, and technological links; and the low-level entities include public opinion and policies. satisfy: , In the above formula, , path The importance of the overall structure is adjusted through the path structure function. The measurement, specifically in the form of: , In the above formula, :path The length of the path The number of edges contained within; , :path The number of high-level and low-level entity nodes; : Penalty coefficient for low-level entity nodes; :path The degree of branching or the number of spare tables it is connected to.

[0026] In one embodiment of the present invention, the penalty coefficient of the low-level entity node It is a constant greater than 1.

[0027] A second aspect of the present invention provides a computer device including a memory, a first processor, and a first computer program stored in the memory and executable on the first processor, wherein the first computer program, when executed by the first processor, implements the aforementioned unified data access orchestration and query execution method.

[0028] A third aspect of the present invention provides a computer-readable storage medium for storing a second computer program, the second computer program being executable by at least one second processor to enable the at least one second processor to perform the above-described unified data access orchestration and query execution method.

[0029] Compared with the prior art, the technical effects achieved by the present invention are as follows: A. To achieve a unified structured expression of complex query logic; B. Supports automatic identification and path reasoning of field attribution; C. Automatically translate query statements and distribute them to the corresponding databases for execution; D. Provide an intermediate layer integration mechanism to achieve the structured return of the final data.

[0030] This invention can effectively reduce the complexity of accessing and managing multiple database systems, improve the flexibility, scalability and development efficiency of data queries, and is applicable to various scenarios such as data middleware, reporting systems, and data service application programming interface (API) platforms. Attached Figure Description

[0031] Figure 1 This is a flowchart of a unified orchestration and query execution method for data access according to an embodiment of the present invention. Detailed Implementation

[0032] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.

[0033] The technical solution of the present invention is illustrated below through specific embodiments. It should be understood that the one or more steps mentioned in the present invention do not preclude the existence of other methods and steps before or after the combined steps, or that other methods and steps may be inserted between these explicitly mentioned steps. It should also be understood that these examples are for illustrative purposes only and are not intended to limit the scope of the present invention. Unless otherwise stated, the numbering of each method step is only for the purpose of identifying each method step, and not for limiting the order of each method or limiting the scope of the present invention. Changes or adjustments to their relative relationships, without substantial changes to the technical content, can also be considered as within the scope of the present invention.

[0034] The raw materials and instruments used in the examples are not subject to any specific restrictions on their source; they can be purchased from the market or prepared according to conventional methods known to those skilled in the art.

[0035] like Figure 1 As shown, the data access unified orchestration and query execution method according to a preferred embodiment of the present invention is applied to a multi-database system (hereinafter referred to as the system), specifically including the following steps.

[0036] S1. Determine the query parameters.

[0037] The query parameters are determined based on the query request provided by the user. The query request must provide structured query parameters, which include the following parameters: Primary key field filtering parameters include: company_id_list (company or enterprise unique identifiers generated internally by the system), where the identifier is the database primary key, such as an auto-incrementing ID or a universally unique identifier (UUID); credit_code_list (company or enterprise unified social credit code), where the code is an 18-digit unique identifier issued by the State Administration for Market Regulation; link_code_list (link codes for internal or external data sources), where the link codes are used for cross-system data connection or relationship identification; and company_name_list (company or enterprise registered name), where the name is the full name registered with the Administration for Industry and Commerce. Primary key field filtering parameters support IN type filtering. Fuzzy matching parameters: Supports fuzzy matching parameters like_name, such as LIKE '%xxx%' in Structured Query Language (SQL) or regular expression matching in MongoDB database; Range-type parameters: such as company_register_capital_list, where the capital is the amount of registered capital registered by the company or enterprise during the business registration, usually in RMB, but may also include foreign currency amounts. Range-type parameters support range queries. Boolean status parameters: such as a dictionary of flags indicating whether company or enterprise-related information exists, i.e., is_exist_dict. The keys of the dictionary are field names or attribute names, and the values ​​are boolean values ​​or integers 0 and 1, used to indicate whether a certain type of information exists. Boolean status parameters support filtering of existence or status-based fields. Pagination and sorting parameters include parameters such as `size` (limiting the number of records returned by the query), `offset` (data offset), `page_number` (page number), `page_size` (number of records per page), and `sort_list` (list of sorting fields). These parameters control the result set. `size` typically has the same meaning as `limit` (limiting the number of records in the query results). `offset` indicates the starting record for the query, used in conjunction with paginated queries. `page_number` indicates the starting page, usually counting from 1. `page_size` controls the size of a single page in paginated queries, and `sort_list` specifies which fields to sort by in ascending or descending order. A page refers to a continuous data unit of records when the database query results contain a large number of records, and these records are divided into multiple pages for gradual display. Each page corresponds to a continuous set of records, defining the range of records read and presented in a single operation. Query field parameter: denoted as find_column, used to represent the list of fields to be returned; The counting control parameter, denoted as track_total_hits, is used to control whether the total number of hits in the search results is calculated precisely. Advanced custom query parameters: such as the nested logic expression parameter sql_custom_query_list, used to support complex nested logic expressions; Array operation parameters: denoted as array_compute_list, used to support array field aggregation or inclusion operations.

[0038] S2, Construction of a unified query model.

[0039] This step parses the user-submitted query parameters into a unified query model that is well-structured, semantically clear, and usable across databases. This lays a standardized foundation for subsequent query adaptation, syntax generation, and execution scheduling. This step includes the following: S21. Extraction of parameter filtering items: Extract all query parameters whose values ​​are not None, that is, the query parameters used to query the true intent of the query request, i.e., the actual query parameters, into filtering conditions, which are simply referred to as conditions. S22. Field semantic recognition: Based on the field corresponding to each actual query parameter (denoted as query field), find its registration information in the field graph, and clarify the database, backup table or document to which it belongs. The field graph refers to a pre-established field meta-information registration structure, which is used to record the semantic information, structural position and database affiliation of all queryable fields, so as to realize the automation of query parameter parsing and the query logic corresponding to the query request. S23. Semantic Type Classification: The conditions are classified into different types, including: exact matching, fuzzy matching, range matching, logical combination, and sorting operations. Exact matching means the field value must exactly match the given value, such as using = or IN. Fuzzy matching is used for partial string matching, such as LIKE or regular expressions. Range matching is used for numerical or time interval queries, such as BETWEEN...AND... or >=, <=. Logical combination refers to multiple conditions combined using logical operators (such as AND, OR, NOT, EXISTS) to form complex conditions. Sorting operations are used to sort query results in ascending or descending order according to one or more fields, usually implemented using ORDER BY. S24. Construction of a unified semantic structure: All conditions corresponding to the user-provided query request are uniformly transformed into a set of intermediate format data bodies. The intermediate format data body represents a composite query condition, wherein the relationship between the conditions is "AND". S25. Custom Logic Tree Fusion: If the query parameter is sql_custom_query_list and the parameter is not empty, merge it with the intermediate format data body to construct a final logical query tree. S26. Logical verification and normalization: Verify the legality of the intermediate format data body, including whether the parameters conflict, whether the data types match, etc., and transform the intermediate format data body into an intermediate abstract query model; S27. Syntax Abstraction Representation Generation: Based on the target database type, the intermediate format data body is mapped to a unified Abstract Syntax Tree (AST) or Semantic Representation Tree as input for subsequent queries.

[0040] The `sql_custom_query_list` is used for custom logical queries. The specific query format can be a query condition tree, which includes a root node, intermediate nodes, and leaf nodes. Each root node, intermediate node, or leaf node represents a condition, which includes a field, an operator, or a target value. The field is a database field, and the operators include four types: < (less than), = (equal to), > (greater than), and != (not equal to). The target value is the desired query result.

[0041] The query condition tree uses the following nested structure to express logical combinations of conditions at any depth: When using sql_custom_query_list for querying, it is implemented through multi-level logical combination. The top-level condition, i.e. the root node of the query condition tree, is a sub-condition that is connected by a logical "OR" relationship with more than one sub-condition, which is called the root sub-condition. One of the root sub-conditions is called the first root sub-condition, which may include three fields: registered province number, registered city number, and registered district / county number. All three fields must meet the corresponding preset values. The three fields are connected by a logical "AND" relationship. Other root sub-conditions are connected to the first condition by a logical "OR" relationship.

[0042] This nested structure has the following characteristics: It supports nested combinations of the logical operators "and" and "or," meaning the nested structure supports arbitrary levels of nesting combinations of the logical operators "and" and "or." Specifically, an "or" node can contain more than one nested "and" child node; an "and" node can also contain multiple nested "or" child nodes. This nested structure supports multi-level recursive combinations and has no limit on nesting depth. From a semantic perspective, this nested structure allows query conditions to express arbitrarily complex logical combinations, possessing strong versatility and extensibility. Through this step, the present invention supports precise expression of traditional SQL, as well as nested query syntax mapped to document databases such as MongoDB and Elasticsearch. It is also compatible with complex nested structures and dynamic condition concatenation, which decouples various query methods and parameter expressions. It also provides a clear and standardized input semantic model for subsequent steps, which is a key step of the present invention.

[0043] S3, Field Attribution Identification and Query Path Modeling.

[0044] This step aims to address the challenges of scattered query field sources, heterogeneous database structures, and difficulties in relational database table joins. It constructs a graph-structured relationship model between query fields and tables or sets in the database (hereinafter collectively referred to as standby tables) to automatically identify field attribution, reason about association paths, and orchestrate multi-source queries, providing fundamental support for complex cross-standby table queries. This step specifically includes the following steps: S31, Field Attribution Analysis.

[0045] First, a global field dictionary is constructed and maintained. This global field dictionary records all fields involved in the query and their metadata, which mainly includes: The field names are the parameters mentioned above, such as: reg_province_id, company_id; The type of data source, such as: MySQL, MongoDB, ClickHouse; Alternate table name, such as: company or enterprise information table; Field types and data types, such as int (integer type), string (string type), array (array type), etc.; Is it a primary key, foreign key, or indexed field? Relationships between primary keys and those of other databases.

[0046] Secondly, when a user submits a set of structured query parameters, each field corresponding to that set of parameters is processed as follows: Field normalization: Identify alternative table names, parameter names, and query fields, and unify them into standard field names; Field attribution location: Find or query the specific database or backup table to which a field belongs; S32, Graph structure modeling.

[0047] A database graph (Field Graph) is constructed and maintained by establishing foreign key relationships between standby tables. This means that the relationships between all standby tables are modeled as a directed graph, which facilitates path reasoning and query plan generation in a multi-database environment.

[0048] The directed graph includes graph nodes and graph edges. Each graph node represents all the required data entities from the databases, i.e., the backup tables, primarily including: tables from relational databases, tables from columnar databases, collections from document-oriented databases, data tables from time-series databases, and entity nodes from graph databases. This invention supports unified modeling of data entities from multiple database types, ensuring complete and accurate query relationships across database platforms. The construction of the directed graph is based on automatic extraction of database metadata.

[0049] There exists a graph edge between every two graph nodes that have a field relationship. This graph edge represents and reflects the field relationship between the two graph nodes. The field relationship includes: foreign key relationships, such as company_id → company.id (the id (identifier) ​​field of the company table, where the companyid field is the primary key of the company table, i.e., it is related to the id field of the company table through the foreign key parameter company_id); and edge attributes. The field relationship is represented by graph edges. The edge attributes include the names of the associated fields, connection type (1:1, 1:N), connection direction, and field attributes of the two graph nodes connected by any graph edge, supporting path reasoning and query execution across backup tables. The relationship between graph nodes is defined by the following parameters: The field name indicates which two fields are associated, e.g., company_id → company.id; the connection type 1:1 indicates that one graph node directly connects to only one other graph node, while connection type 1:N indicates that one graph node directly connects to N other graph nodes, where N is an integer greater than 1; the connection direction indicates which graph node's field points to the corresponding field in the other graph node; and field attributes include data type and constraints. These edge attributes allow for the automatic derivation of the relationship paths between data fields along the graph edges. In other words, when querying data involving multiple standby tables or databases, the connection order and method between data can be determined based on the graph edge attributes, thus enabling cross-standby table joins.

[0050] In a directed graph, data entities are mapped as graph nodes, and foreign key relationships are mapped as directed edges, i.e., graph edges. The direction of the graph edges points from the foreign key to the corresponding primary key. Each graph edge record, or association type, includes a connection field. Maintaining a directed graph requires ensuring the integrity of the graph structure and reflecting the structural relationships of the database in real time.

[0051] The field associations refer to the logical or physical connections established between data entities represented by two graph nodes through fields. These include explicit primary key-foreign key constraints declared in the database and implicit associations inferred based on business semantics. For example, the foreign key association between the field `company_id` and the primary key `id` of the table `company` is an explicit association; the business correspondence between the field `reg_province_id` (the `id` field of the Chinese province and city database table `reg_province`) and the primary key `code` (province code) of the province code table `province` is an implicit association.

[0052] In this invention, after querying the fields corresponding to the parameters, field attribution resolution is used to determine which graph node the field belongs to, thereby realizing the association mapping from the fields corresponding to the query parameters to the graph nodes. This supports subsequent path reasoning and query planning based on the graph structure. This mapping ensures that the query request can accurately locate the data and realize intelligent query execution across backup tables based on the graph structure.

[0053] S33, Graph Path Reasoning and Query Relationship Planning.

[0054] A directed graph should be able to handle multi-field queries. For example, consider a query that uses fields such as company_name, industry_code, and reg_city, which may belong to different tables. The following steps are used to infer the join path in the directed graph: S331. Assign and locate multiple fields to obtain their respective graph nodes; S332. Calculate different connection paths between the corresponding graph nodes in the directed graph, such as the shortest path, the path with the fewest hops, and the path with the lowest cost. This step is used to determine the optimal access method between different data entities. The shortest path is the path that connects two graph nodes with the fewest graph edges, that is, the path with the fewest hops and the simplest access steps. The path with the fewest hops is the path that passes through the fewest graph nodes from the starting graph node to the ending graph node. This path has a similar meaning to the shortest path, both of which are to reduce the complexity and cost of accessing across spare tables. The path with the lowest cost is the path with the lowest overall overhead, which comprehensively considers access costs, including factors such as data size, network latency, and computing resource consumption. Through the above path calculation, the optimal data access scheme can be selected to improve query efficiency and system performance. S333. For each field in the multi-field, generate a directed acyclic graph (DAG) of the subquery path, determine the dependency order of data access at each step, and then arrange the INNER JOIN or nested execution logic of the subquery accordingly.

[0055] The field graph supports dynamic registration and updates, and new standby tables, fields, or relationships can be added to the graph structure in real time; it supports automatic graph structure construction through configuration files or database models; and it supports the injection of custom business logic paths, such as adding special field relationship rules based on industry experience.

[0056] By using field attribution parsing and graph structure modeling, this invention achieves automatic identification of field positions in cross-database queries, intelligent planning of association paths, and automatic generation of access processes, thereby significantly improving query organization efficiency, maintainability, and automation levels in multi-data source systems.

[0057] It should be noted that when planning the query relationship in response to a user's query request, it is based on the database graph, i.e., the directed graph, which enumerates all possible query paths and selects the optimal path as the basis for the execution plan to respond to the user's query request.

[0058] S4. Expected Path Reasoning and Selection.

[0059] In database graph-based query planning, it is necessary to enumerate all possible query paths within an entity-relationship network based on the database graph, and select the optimal path as the basis for the execution plan based on a strategy. This step specifically includes the following steps: S41. Global Data Relationship Graph Modeling and Entity Relationship Abstraction. The global data relationship graph is the data relationship network corresponding to the relationships between entities in the query object. Essentially, this step maintains a global data relationship graph based on the database graph, where: The global data relationship graph includes more than one node and multiple edges. Each node corresponds one-to-one with a graph node in the database graph, representing a spare table in a query object, such as company, region, or industry. The entities represent various types of entities involved in enterprise-level data, such as enterprises, projects, personnel, industries, and regions. The edges represent the logical relationships between these entities, such as affiliation, classification, association, and upstream / downstream dependencies. Each edge has a direction attribute (directed or undirected) and includes various semantic labels, time labels, source information, and credibility scores. Each of the multiple edges has a one-to-one corresponding graph edge in the database graph.

[0060] S42, Multidimensional path weight modeling and quantitative evaluation.

[0061] To achieve accurate path selection and ranking, this invention introduces an adjustable multidimensional path cost evaluation function, assigning a modified weight to each edge in the global data relationship graph, denoted as... For any given path between any two specified nodes in a global data relationship graph, calculate its path score. The calculation formula is as follows: , In the above formula, Represents the first connection in the global data relationship graph. i Nodes and the j Nodes edge The corrected weights reflect the semantic importance of the nodes it connects to, where, i and j All are integers; For path The path structure adjustment function is used for overall path evaluation. The structural complexity, the rationality of node distribution, etc.

[0062] Remember the edge The basic weight is , The weighting can be determined based on factors such as time decay, credibility, and source level. The expression is as follows: , For node pairs and The type adjustment function is used to reflect the edge The importance level of the connected nodes is called the node level. The node entity corresponding to the node level, that is, the entity included or associated with the fields in the node, is a high-level entity, a medium-level entity, or a low-level entity. For example, enterprises and patents are high-level entities, industrial chains, industrial links, technologies, and technological links are medium-level entities, and public opinion and policies are low-level entities. satisfy: , In the above formula, .

[0063] path The importance of the overall structure is adjusted through the path structure function. The measurement, specifically in the form of: , In the above formula, :path Length (i.e., path) (Number of edges contained); , :path The number of high-level and low-level entity nodes; The penalty coefficient for low-level entity nodes is preferably a constant greater than 1; :path The degree of branching or the number of spare tables connected to the path is used to encourage information-integrated paths.

[0064] Users rate based on path Path sorting can obtain the optimal path between any two specified nodes in a given global data relationship graph.

[0065] The present invention also provides a computer device, which includes a memory, a first processor, and a first computer program stored in the memory and executable on the first processor. When the first computer program is executed by the first processor, it implements the above-mentioned unified data access orchestration and query execution method.

[0066] The present invention also provides a computer-readable storage medium for storing a second computer program, which can be executed by at least one second processor to enable the at least one second processor to perform the above-described unified data access orchestration and query execution method.

[0067] This invention achieves a unified structured expression of complex query logic, supports automatic identification of field attribution and path reasoning, and can automatically translate query statements and distribute them to the corresponding databases for execution. It provides an intermediate layer integration mechanism to achieve the final structured return of data. This invention effectively reduces the complexity of accessing and managing multiple database systems, improves the flexibility, scalability, and development efficiency of data queries, and is applicable to various scenarios such as data platforms, reporting systems, and data service API platforms.

[0068] The foregoing description of specific exemplary embodiments of the invention is for illustrative and explanatory purposes. These descriptions are not intended to limit the invention to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of the invention and its practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of the invention, as well as various different choices and variations. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A unified orchestration and query execution method for data access, characterized in that, Including the following steps: S1. Determine the query parameters, that is, determine the query parameters based on the query request provided by the user. The query request must provide the structured query parameters. S2. Construction of a unified query model, which involves parsing the query parameters into a unified query model for use across databases; S3. Field attribution identification and query path modeling, namely, constructing a graph structure relationship model between the fields corresponding to the query parameters and the backup table, wherein the backup table includes at least one of the following data entities in the database: table, set; S4. Expected path reasoning and selection, that is, enumerating all query paths and selecting the optimal query path based on the strategy.

2. The unified data access orchestration and query execution method according to claim 1, characterized in that, The query parameters include: primary key field filtering parameters, fuzzy matching parameters, range parameters, boolean status parameters, pagination and sorting parameters, query field parameters, count control parameters, advanced custom query parameters, and array operation parameters.

3. The unified orchestration and query execution method for data access according to claim 1 or 2, characterized in that, Step S2 includes: S21. Parameter filtering item extraction: Extract all query parameters used to query the true intent of the query request, i.e., the actual query parameters, into filtering conditions, or simply conditions. S22, Field semantic recognition: Based on the field corresponding to each actual query parameter, find its registration information in the field graph, and determine the database, spare table or document to which it belongs. The field graph refers to a pre-established field meta-information registration structure used to record the semantic information, structural position and database affiliation of all queryable fields. S23. Semantic type classification: Divide each of the conditions into different types, and the different types include at least one of the following types: exact matching, fuzzy matching, range matching, logical combination, and sorting operation; S24. Construction of a unified semantic structure: All the conditions are uniformly converted into a set of intermediate format data bodies, wherein the intermediate format data body represents a composite query condition, and the relationship between the conditions is AND; S26. Logical verification and normalization: Verify the legality of the intermediate format data body and transform the intermediate format data body into an intermediate abstract query model; S27. Syntax Abstraction Representation Generation: Based on the target database type, the intermediate format data body is mapped into a unified abstract syntax tree or semantic representation tree, which serves as the input for subsequent queries.

4. The unified orchestration and query execution method for data access according to claim 3, characterized in that, Step S2 further includes the following steps: S25. Custom Logic Tree Fusion: If the query parameter is sql_custom_query_list and it is not empty, merge it with the intermediate format data body to construct a final logic query tree.

5. The unified orchestration and query execution method for data access according to claim 4, characterized in that, Step S3 includes: S31, Field attribution resolution, including: First, a global field dictionary is constructed and maintained. This global field dictionary records all fields involved in the query and their metadata, including: The field name refers to each query parameter, the data source type, the alternate table name, the field type and data type, whether it is a primary key, foreign key or index field, and the relationship with the primary keys of other databases; Secondly, when a user submits a set of structured query parameters, each field corresponding to the set of structured query parameters is processed as follows: Field normalization and field attribution; S32. Graph structure modeling, including: A database graph is constructed and maintained through the foreign key relationships between the standby tables. This involves modeling the relationships between all the standby tables as a directed graph, i.e., the database graph. The directed graph includes graph nodes and graph edges. The graph nodes represent all the required data entities in the database, i.e., the standby tables, including: tables in relational databases, tables in columnar databases, collections in document-oriented databases, data tables in time-series databases, and entity nodes in graph databases. There is a graph edge between every two graph nodes that have a field relationship. The graph edge reflects the field relationship between the two graph nodes. The field relationship includes: foreign key relationship and edge attributes. The edge attributes include the associated field name, connection type (1:1, 1:N), connection direction, and field attributes of the two graph nodes connected by any graph edge. The associated field name indicates which two fields are associated. The connection type includes 1:1 and 1:N. The 1:1 connection type means that one graph node directly connects to only one other graph node. The 1:N connection type means that one graph node directly connects to N other graph nodes, where N is an integer greater than 1. The connection direction indicates which field of the two graph nodes points to the corresponding field of the other graph node. The field attributes include: the data type and constraints of the field. S33, Graph Path Reasoning and Query Relationship Planning.

6. The unified orchestration and query execution method for data access according to claim 5, characterized in that, Step S33 includes: When the directed graph is used to process multi-field queries, the method for reasoning connection paths in the directed graph includes: S331. Assign and locate the multiple fields to obtain their respective graph nodes; S332. Calculate the different connection paths between the graph nodes obtained in step S331 in the directed graph to determine the optimal access method between different data entities; S333. For each of the multiple fields, generate a directed acyclic graph of subquery paths to determine the dependency order of data access at each step.

7. The unified orchestration and query execution method for data access according to claim 6, characterized in that, Step S4 includes: S41. Global data relationship graph modeling and entity relationship abstraction: This involves maintaining a global data relationship graph based on the database graph. The global data relationship graph corresponds to the data relationship graph between entities in the query object. The global data relationship graph includes multiple nodes and multiple edges. Each node corresponds one-to-one with a graph node in the database graph, representing a spare table in a query object. Each edge represents a logical relationship between different entities. The logical relationship includes at least one of the following relationships: membership, classification, association, and upstream / downstream dependency. Each edge has a direction attribute: directed or undirected, and includes multiple semantic labels, time labels, source information, and credibility scores. Each edge in the multiple edges has a one-to-one corresponding graph edge in the database graph. S42. Multidimensional path weight modeling and quantitative evaluation remember For any given path between any two specified nodes in the global data relationship graph, its path score is calculated. The calculation formula is as follows: ， In the above formula, Represents the first connection in the global data relationship graph. i Nodes and the j Nodes edge The corrected weights, where, i and j All are integers; For path The path structure adjustment function, Remember the edge The base weight is , The weights are adjusted based on time decay, credibility, and source level. The expression is as follows: ， For node pairs and The type adjustment function is used to reflect the edge The importance level of the connected nodes is called the node level. The node entity corresponding to the node level, that is, the entity included or associated with the fields in the node, is a high-level entity, a medium-level entity, or a low-level entity. The high-level entities include enterprises and patents; the medium-level entities include industrial chains, industrial links, technologies, and technological links; and the low-level entities include public opinion and policies. satisfy: ， In the above formula, ， path The importance of the overall structure is adjusted through the path structure function. The measurement, specifically in the form of: ， In the above formula, :path The length of the path The number of edges contained within; , :path The number of high-level and low-level entity nodes; : Penalty coefficient for low-level entity nodes; :path The degree of branching or the number of spare tables it is connected to.

8. The unified orchestration and query execution method for data access according to claim 7, characterized in that, The penalty coefficient of the low-level entity node It is a constant greater than 1.

9. A computer device, characterized in that, It includes a memory, a first processor, and a first computer program stored in the memory and executable on the first processor. When the first computer program is executed by the first processor, it implements the unified orchestration and query execution method for data access as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a second computer program, which can be executed by at least one second processor to enable the at least one second processor to perform the unified orchestration and query execution method for data access as described in any one of claims 1-8.

Images