A database query method, apparatus, device and medium

Abstract

This application discloses a database query method, apparatus, device, and medium, belonging to the field of data retrieval. The method comprises: determining the business domain label of each metadata element based on the acquired metadata of the database to be queried; constructing a two-level inverted index based on the business domain label; wherein the two-level inverted index includes a table index and a metadata index; receiving a query request, obtaining the first tenant identifier and the query business domain associated with the query request, and matching in the table index according to the first tenant identifier and the first association relationship to determine several candidate data tables; using all candidate data tables and the query business domain as range filtering conditions, performing metadata retrieval at each index level according to the second association relationship in the order of retrieval priority, until the target metadata that meets the preset confidence conditions is obtained, and outputting the target metadata. This application can improve the metadata query speed in data storage scenarios with multiple tenants and multiple services coexisting.

Description

Technical Field

[0001] This application relates to the field of data retrieval, and in particular to a database query method, apparatus, device, and medium. Background Technology

[0002] As enterprises continuously improve their informatization and digitalization, the scale of data in databases continues to grow. In multi-tenant architectures and cross-business system collaboration scenarios, databases often simultaneously host data tables and their metadata from different business domains and tenants. When performing database queries, users increasingly rely on natural language or business semantics-driven methods, hoping to quickly locate target metadata that highly matches their current business semantics from massive database objects. Therefore, how to efficiently organize and retrieve metadata based on business semantics in complex database environments has become a fundamental problem that urgently needs to be solved in the field of data retrieval.

[0003] Existing database query technologies typically rely on structured field matching or single-level index retrieval. While these technologies support keyword search and simple semantic expansion to some extent, in multi-tenant, multi-business domain scenarios, metadata often lacks clear business domain distinctions. Metadata from different tenants and with different business semantics is mixed, stored, and retrieved uniformly. This results in queries needing to traverse and match through a large amount of irrelevant metadata, leading to lengthy and inefficient retrieval paths and difficulty in quickly focusing on metadata highly relevant to the current query's business domain. Therefore, achieving fast metadata retrieval in multi-tenant, multi-business data storage scenarios is a key technical problem that needs to be solved. Summary of the Invention

[0004] This application provides a database query method, apparatus, device, and medium that can solve the problem of how to achieve fast metadata query in data storage scenarios with multiple tenants and multiple services coexisting in the prior art.

[0005] Some embodiments of this application provide a database query method, including: Based on the acquired metadata of the database to be queried, a business domain tag for each piece of metadata is determined, and a two-level inverted index is constructed according to the business domain tag. The two-level inverted index includes a table index and a metadata index. The table index stores a first association between the tenant identifier and the data table. The metadata index stores a second association between the metadata and the business domain tag, and the metadata index is divided into multiple index levels with different retrieval priorities according to the access characteristics of the corresponding metadata. Receive a query request, obtain the first tenant identifier and query business domain associated with the query request, and match the table index according to the first tenant identifier and the first association relationship to determine several candidate data tables; Using all the candidate data tables and the query business domain as range filtering conditions, metadata retrieval is performed at each of the index levels according to the second association relationship in the order of the retrieval priority until the target metadata that meets the preset confidence conditions is obtained, and the target metadata is output.

[0006] Compared to existing technologies, the above embodiments have the following beneficial effects: This application distinguishes database metadata by business domain tags and constructs a two-level inverted index structure containing table indexes and metadata indexes based on the business domain tags, enabling metadata to be logically organized in an orderly manner according to the tenant dimension and business semantic dimension. When receiving a query request, the tenant identifier is first used to limit the range of candidate data tables in the table index, and then the range of metadata retrieval is constrained by the query business domain, thereby avoiding indiscriminate retrieval of all database objects and significantly reducing the retrieval space. Furthermore, by introducing multi-level retrieval priorities based on access characteristics into the metadata index, and executing metadata retrieval level by level according to priority during the retrieval process, the query process can prioritize matching in the highly relevant index level. The retrieval process can be terminated early after the pre-set confidence condition is met, reducing unnecessary index access and computation overhead. Ultimately, it achieves targeted and efficient querying of metadata in multi-tenant, multi-business coexistence scenarios, improving query response speed and overall system retrieval efficiency.

[0007] Furthermore, the metadata index is divided into multiple index levels with different retrieval priorities based on the access characteristics of the corresponding metadata, including: The number of times the metadata is retrieved within a preset period is counted, and the popularity score of the metadata is determined based on the number of times it is retrieved. The retrieval priority of the metadata is determined based on the popularity score, and the corresponding metadata index is divided into the corresponding index level according to the retrieval priority.

[0008] Compared to existing technologies, the above embodiments offer the following advantages: By introducing access frequency to quantitatively model the access characteristics of metadata, the retrieval priority of the metadata index can be determined. This approach not only reflects the frequency of metadata usage in historical queries but also dynamically reflects the recent activity of metadata, allowing the index hierarchy to adapt to actual query behavior rather than relying on static rules. Therefore, in multi-tenant, multi-service, high-concurrency query scenarios, high-value, highly relevant metadata can be prioritized for entry into high-retrieval-priority index layers, fundamentally reducing the interference of low-relevant metadata on the query path and further shortening the average retrieval time of metadata.

[0009] Further, the index hierarchy includes: a frequently used table index layer, a regular index layer, and a cold data index layer; the step of determining the retrieval priority of the metadata based on the popularity score, and dividing the corresponding metadata index into the corresponding index hierarchy according to the retrieval priority, includes: When the number of times the metadata corresponding to the metadata index is retrieved is greater than a preset frequency, the score range to which the popularity score of the metadata index belongs is determined. If the popularity score is greater than the first preset score, the metadata index will be assigned to the commonly used table index layer with the first retrieval priority. If the popularity score is greater than the second preset score but not greater than the first preset score, then the metadata index is assigned to the ordinary index layer with the second retrieval priority. If the popularity score is not greater than the second preset score, the metadata index will be assigned to the cold data index layer with the third retrieval priority.

[0010] Compared to existing technologies, the above embodiments offer the following advantages: Based on determining retrieval priority according to popularity scores, the metadata index is explicitly divided into a frequently used table index layer, a regular index layer, and a cold data index layer. Layering is determined by score ranges, ensuring stable and clear hierarchical boundaries for metadata with different access characteristics within the index structure. This layering approach avoids the disordered scanning problem caused by the mixed storage of metadata with varying popularity during retrieval, enabling the system to prioritize matching in the frequently used table index layer during metadata retrieval, thereby reducing the frequency of access to the low-activity index layer. In database environments with multiple concurrent services, this layering mechanism helps reduce the drag on query efficiency from cold data, improving the determinism and predictability of overall metadata queries.

[0011] Further, the step of using all the candidate data tables and the query business domain as range filtering conditions, and performing metadata retrieval at each of the index levels according to the second association relationship in order of retrieval priority, includes: Based on the query request, negative semantic keywords are generated, and filtering instructions are generated based on the negative semantic keywords; When metadata retrieval is performed sequentially in each of the index levels, the filtering instructions are used to perform semantic consistency checks on each of the metadata associated in the current index level, and the metadata indexes corresponding to the metadata that match the meaning of the negative semantic keywords are removed to obtain the first index set; The first index set is filtered by all the candidate data tables and the query business domain to obtain the second index set, and metadata retrieval of the corresponding index level is performed in the second index set according to the second association relationship.

[0012] Compared to existing technologies, the above embodiments have the following advantages: Addressing the issue that relying solely on business domain filtering may still introduce semantically irrelevant metadata, this method introduces negative semantic keywords to generate filtering instructions and performs semantic consistency checks during the retrieval process at each index level, thus enabling the early removal of metadata indexes that do not conform to query semantic constraints. This method limits metadata retrieval not only to the table scope and business domain scope but also further to the exclusive constraints of query semantics, thereby reducing the probability of invalid matches within the index level. By moving the semantic filtering stage forward in the retrieval phase, the size of candidate indexes can be effectively reduced, allowing subsequent metadata retrieval to focus more on semantically relevant objects, improving retrieval efficiency and accuracy in complex business semantic query scenarios.

[0013] Furthermore, obtaining the target metadata that meets the preset confidence conditions includes: The metadata associated with the second index set is used as the first candidate metadata. The consistency between the first candidate metadata and the query request is evaluated to obtain the semantic relevance of each first candidate metadata. When metadata retrieval is performed sequentially in each of the index levels, the level weight corresponding to the current index level is obtained. By weighting the level weight and the semantic relevance, the comprehensive confidence score of each of the first candidate metadata in the current index level is obtained. If there are several second candidate metadata whose overall confidence score is greater than a preset threshold among all the first candidate metadata in the current index level, then the second candidate metadata with the largest overall confidence score will be used as the target metadata.

[0014] Compared to existing technologies, the above embodiments have the following advantages: Based on the second index set, by evaluating the consistency between candidate metadata and query requests, semantic relevance is introduced, and a comprehensive confidence score is calculated by combining index level weights, thereby achieving refined sorting and filtering of different candidate metadata. This approach ensures that the selection of target metadata no longer relies solely on a single level or a single semantic similarity, but comprehensively considers the semantic matching degree of the metadata itself and the importance of its index level. When high-confidence metadata is identified in advance in a high-priority index layer, the subsequent retrieval process can be directly terminated, avoiding invalid traversal, further compressing query response time, and improving the ability to quickly locate metadata.

[0015] Further, determining the business domain label for each piece of metadata based on the acquired metadata of the database to be queried includes: By using preset filtering rules, the first metadata that does not have business semantics is removed from all the metadata to obtain the third candidate metadata; An enhanced semantic description of the third candidate metadata and several corresponding distinguishing data with the same semantics are generated using a pre-defined large language model. Obtain the context information of the third candidate metadata in the database to be queried, and determine the business domain label of each third candidate metadata based on the context information, the enhanced semantic description, and all the distinguishing data.

[0016] Compared to existing technologies, the above embodiments offer the following advantages: Addressing the issue of significant business semantic noise in the original metadata structure, this approach removes metadata lacking business semantics through preset filtering rules. It then generates enhanced semantic descriptions and semantically equivalent distinguishing data using a large language model, and further determines business domain labels by incorporating contextual information. This processing method improves the semantic accuracy and coverage of business domain labels during the index building stage, making subsequent business domain-based index filtering more precise and reliable. Consequently, it reduces cross-business false detections caused by ambiguous or missing business domain labels, providing a more robust semantic foundation for rapid metadata retrieval in multi-business scenarios.

[0017] Further, receiving the query request and obtaining the first tenant identifier and query business domain associated with the query request includes: Obtain the access token carried in the query request, and parse the access token carried in the query request to obtain the first tenant identifier; The core business keywords are extracted from the query request and matched with a preset business domain knowledge base to determine the query business domain to which the query request belongs.

[0018] Compared to existing technologies, the above embodiments offer the following advantages: By parsing the access token carried in the query request to determine the tenant identifier, and matching the core business keywords in the query request with the business domain knowledge base, the query business domain is automatically determined, thereby avoiding reliance on manually specifying the tenant or business scope. This approach enables the accurate and automatic extraction and injection of tenant information and business semantics from the query context into the metadata retrieval process, ensuring the accuracy and reliability of the preconditions for subsequent table index matching and business domain filtering. In a multi-tenant, multi-business environment, this mechanism helps reduce the risk of erroneous queries across tenants and businesses, making the metadata query process more efficient and consistent with the actual business context.

[0019] Another embodiment of this application also provides a database query device, including: an index building module, a table filtering module, and a data retrieval module; The index building module is used to determine the business domain label of each piece of metadata based on the acquired metadata of the database to be queried, and to build a two-level inverted index based on the business domain label; wherein, the two-level inverted index includes: a table index and a metadata index; the table index stores a first association relationship between the tenant identifier and the data table; the metadata index stores a second association relationship between the metadata and the business domain label, and the metadata index is divided into multiple index levels with different retrieval priorities according to the access characteristics of the corresponding metadata; The table filtering module is used to receive a query request, obtain the first tenant identifier and query business domain associated with the query request, and match the table index according to the first tenant identifier and the first association relationship to determine several candidate data tables. The data retrieval module is used to perform metadata retrieval at each of the index levels according to the second association relationship, based on the second association relationship, using all the candidate data tables and the query business domain as range filtering conditions, and in the order of the retrieval priority, until the target metadata that meets the preset confidence conditions is obtained, and then outputs the target metadata.

[0020] Another embodiment of this application provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of the database query method of this application.

[0021] Another embodiment of this application also provides a computer-readable storage medium item, including: a stored computer program, which, when the computer program is running, controls the device where the computer-readable storage medium is located to perform steps such as the database query method of this application. Attached Figure Description

[0022] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a database query method provided in some embodiments of this application; Figure 2 This is a schematic diagram of the structure of a database query device provided in some embodiments of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0026] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0027] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0028] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0029] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0030] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0031] Existing database query technologies typically rely on structured field matching or single-level index retrieval. While they support keyword search or simple semantic expansion to some extent, in scenarios with multiple tenants and multiple business domains, metadata often lacks clear distinction between business domains. Metadata from different tenants and different business semantics is mixed, stored, and retrieved uniformly. This results in queries needing to traverse and match a large amount of irrelevant metadata, leading to lengthy and inefficient retrieval paths and difficulty in focusing on metadata highly relevant to the current query's business domain.

[0032] Please refer to Figure 1 To address the issue of improving metadata query speed in multi-tenant, multi-service data storage scenarios, this application provides a database query method comprising the following steps S101 to S103: S101: Based on the acquired metadata of the database to be queried, determine the business domain label of each piece of metadata, and construct a two-level inverted index according to the business domain label; wherein, the two-level inverted index includes: a table index and a metadata index; the table index stores the first association relationship between the tenant identifier and the data table; the metadata index stores the second association relationship between the metadata and the business domain label, and the metadata index is divided into multiple index levels with different retrieval priorities according to the access characteristics of the corresponding metadata.

[0033] Preferably, in some embodiments of this application, the metadata consists of fields stored in a database, each field corresponding to: field name, field type, field comment information, the database name where the field resides, and the table name where the field resides. It is understood that because some fields lack field comment information, or the field comment information is too brief to directly reflect their specific business semantics, this may affect the subsequent determination of the business domain label corresponding to the field.

[0034] Furthermore, in some embodiments of this application, determining the business domain label for each piece of metadata based on the acquired metadata of the database to be queried includes: By using preset filtering rules, the first metadata that does not have business semantics is removed from all the metadata to obtain the third candidate metadata; An enhanced semantic description of the third candidate metadata and several corresponding distinguishing data with the same semantics are generated using a pre-defined large language model. Obtain the context information of the third candidate metadata in the database to be queried, and determine the business domain label of each third candidate metadata based on the context information, the enhanced semantic description, and all the distinguishing data.

[0035] Preferably, in some embodiments of this application, the filtering rules include at least: rule set R1, rule set R2 and rule set R3, each rule set is independent of the others and is executed using deterministic conditions.

[0036] Rule set R1 filters fields that serve only as technical identifiers, internal numbers, or unique constraints. These fields are typically not used as business attributes in semantic modeling and keyword retrieval. A field is removed from rule set R1 if it meets at least one of the following conditions: the field is defined as a primary key or used to represent an internal unique identifier; the field semantically represents only technical meanings such as "number," "ID," or "identifier"; or the field is primarily used for data association or unique constraints, rather than business metrics or attributes. For example, in a data table, a field of integer type is defined as a primary key to represent the uniqueness of a record, without carrying specific business meaning. In this case, the field meets the condition "the field is defined as a primary key or used to represent an internal unique identifier" in rule set R1, and is therefore identified as primary metadata and removed.

[0037] Rule set R2 is used to filter template fields, placeholder fields, or extended reserved fields introduced during the database design process. These fields typically do not carry explicit business semantics. A field is removed from rule set R2 if it meets at least one of the following conditions: the field name matches the predefined set of template field names; the field comment is empty, or the comment content is only a non-business description such as "default field" or "placeholder field"; the combination of the field name and field type cannot be mapped to a specific business object or business attribute. For example, in a certain data table, there is a field of variable length string type that does not contain any business-oriented field name or comment information and exists only as a placeholder or extended field. The above field matches the "template and placeholder fields" filtering condition in rule set R2 and is therefore removed as first metadata.

[0038] Rule set R3 is used to filter fields in the database that are used for system operation control, status identification, or audit records. These fields have fixed and stable semantics and are generally not used as objects for user queries or semantic modeling. A field is removed from rule set R3 when it meets at least one of the following conditions: the field is used for logical deletion or status control; the field is used to record audit information such as data creation time and update time; the field is used only for system maintenance or operation management. For example, a field in a data table is used to indicate logical deletion status, or a field is used to record the creation or update time of data. These fields all belong to system control or audit fields, match rule set R3, and are therefore uniformly removed as primary metadata.

[0039] For example, the third candidate metadata obtained after filtering out the first metadata can be the field amt, where amt is the field name and the corresponding annotation information for amt is "bill amount". It can be seen that this field has specific business semantics, so it is necessary to further determine its corresponding business domain label.

[0040] Preferably, in some embodiments of this application, the process of generating the enhanced semantic description includes: inputting the field name, field type, field annotation information, and context information in the data table containing the field, corresponding to the third candidate metadata, into a preset large language model, and performing field-level semantic reasoning to generate the enhanced semantic description. During the generation of the enhanced semantic description, different semantic enhancement operations are performed based on the absence or simplicity of the field annotation information, including: if the field annotation information is empty, a complete semantic description is generated; if the field annotation information is a weak semantic annotation (such as "type" or "number"), semantic expansion is performed while retaining the original annotation meaning; if the field annotation information already possesses clear business semantics, it can be directly reused or lightly enhanced.

[0041] Preferably, in some embodiments of this application, the distinguishing data includes: Chinese aliases and English aliases corresponding to the field names. The generation process of the distinguishing data includes: generating multiple Chinese aliases and English aliases for the field using a large language model; wherein, the Chinese aliases are used to cover the natural language expressions of different business personnel; the English aliases are used to cover field abbreviations, English queries, and mixed Chinese and English query scenarios; the aliases are consistent with the semantic description of the field and do not introduce ambiguous meanings. For example, for the field amt, its Chinese alias is bill amount or payable amount, and its English alias is amount or bill_amount.

[0042] Preferably, in some embodiments of this application, the process of determining the business domain label of each of the third candidate metadata includes: performing business domain classification processing on the third candidate metadata through a large language model based on the enhanced field semantic description (i.e., enhanced semantic description), the field alias set (i.e., the set of distinguishing data), and the context information of the data table where the third candidate metadata is located, thereby assigning a business domain label to each third candidate metadata. The business domain label is used to represent the business domain to which the corresponding third candidate metadata belongs; multiple different third candidate metadata in the same data table can be assigned to the same business domain. For example, for the third candidate metadata amt, it is assigned to the business domain "billing".

[0043] Preferably, in some embodiments of this application, the table index includes: a tenant identifier, the database name of the database to which the data table belongs, the table name, the table's comment information, an alias for the table name, and business domain tags present in the data table. Therefore, the table index reveals the first association between the tenant identifier and the data table. As can be seen from the structure of the table index, it is used for cataloging databases and data tables, supporting the location of candidate data tables during queries. Its index granularity is "table," used to answer the question "which databases and data tables might correspond to the given query request?"

[0044] Preferably, in some embodiments of this application, the metadata index includes: tenant identifier, business domain tag of the metadata, database name of the database to which the metadata belongs, table name of the data table to which the metadata belongs, storage path of the metadata, field name of the metadata, enhanced semantic description of the metadata, alias of the metadata, and semantic confidence of the metadata. Therefore, through the metadata index, the relationship between each piece of metadata and the business domain tag can be known. As can be seen from the structure of the above metadata index, the metadata index is used to store standardized structured entries at the field level and constructs an inverted index with dimensions such as field name, semantic description, and field alias to support metadata retrieval based on keywords and natural language.

[0045] As can be seen from the above preferred embodiments, since the metadata stored in each data table is known, the business domain tags included in each data table can also be known based on the metadata index. Thus, the batch of metadata corresponding to the business domain to be queried can be quickly located through the table index.

[0046] To address the issue of significant business semantic noise in the original metadata structure, this approach removes metadata lacking business semantics through pre-defined filtering rules. It then combines this with a large language model to generate enhanced semantic descriptions and semantically equivalent distinguishing data, further integrating contextual information to determine business domain labels. This process improves the semantic accuracy and coverage of business domain labels during the index building phase, making subsequent business domain-based index filtering more precise and reliable. Consequently, it reduces cross-business false detections caused by ambiguous or missing business domain labels, providing a more robust semantic foundation for rapid metadata retrieval in multi-business scenarios.

[0047] Furthermore, in some embodiments of this application, the metadata index is divided into multiple index levels with different retrieval priorities based on the access characteristics of the corresponding metadata, including: The number of times the metadata is retrieved within a preset period is counted, and the popularity score of the metadata is determined based on the number of times it is retrieved. The retrieval priority of the metadata is determined based on the popularity score, and the corresponding metadata index is divided into the corresponding index level according to the retrieval priority.

[0048] By introducing access frequency to quantitatively model the access characteristics of metadata, the retrieval priority of the metadata index is determined. This approach not only reflects the frequency of metadata usage in historical queries but also dynamically reflects the recent activity of metadata. This allows the index hierarchy to adapt to actual query behavior, rather than relying on static rules. Therefore, in multi-tenant, multi-service, high-concurrency query scenarios, high-value, highly relevant metadata can be prioritized for entry into high-retrieval-priority index layers, fundamentally reducing the interference of low-relevant metadata on the query path and further shortening the average retrieval time of metadata.

[0049] Preferably, in some embodiments of this application, the formula for calculating the heat score is: ; in, For metadata In the preset period The heat score within; For metadata In the preset period The number of times it was retrieved; For all metadata within a preset period The maximum number of times it has been retrieved.

[0050] Furthermore, in some embodiments of this application, the index hierarchy includes: a frequently used table index layer, a regular index layer, and a cold data index layer; the step of determining the retrieval priority of the metadata based on the popularity score, and dividing the corresponding metadata index to the corresponding index hierarchy according to the retrieval priority, includes: When the number of times the metadata corresponding to the metadata index is retrieved is greater than a preset frequency, the score range to which the popularity score of the metadata index belongs is determined. If the popularity score is greater than the first preset score, the metadata index will be assigned to the commonly used table index layer with the first retrieval priority. If the popularity score is greater than the second preset score but not greater than the first preset score, then the metadata index is assigned to the ordinary index layer with the second retrieval priority. If the popularity score is not greater than the second preset score, the metadata index will be assigned to the cold data index layer with the third retrieval priority.

[0051] Preferably, in some embodiments of this application, if the number of times the second metadata is retrieved is less than a preset frequency, then there is no need to semantically enhance the second metadata and determine the business domain tag corresponding to the second metadata. Furthermore, there is no need to construct a metadata index corresponding to the second metadata, nor is there a need to construct a table index for the data table to which the second metadata belongs. The second metadata is directly stored in the corresponding static data layer. It should be noted that this static data layer and each index level are used for retrieving metadata.

[0052] Based on the principle of prioritizing retrieval according to popularity scores, the metadata index is explicitly divided into a frequently used table index layer, a regular index layer, and a cold data index layer. Layering is determined by score ranges, ensuring stable and clear hierarchical boundaries for metadata with different access characteristics within the index structure. This layering approach avoids the unordered scanning problem caused by the mixed storage of metadata with varying popularity during retrieval, allowing the system to prioritize matching in the frequently used table index layer during metadata retrieval, thereby reducing the frequency of access to the low-activity index layer. In a database environment with multiple concurrent services, this layering mechanism helps reduce the drag on query efficiency caused by cold data, improving the determinism and predictability of overall metadata queries.

[0053] S102: Receive a query request, obtain the first tenant identifier and query business domain associated with the query request, and match the table index according to the first tenant identifier and the first association relationship to determine several candidate data tables.

[0054] Furthermore, in some embodiments of this application, the step of receiving a query request and obtaining the first tenant identifier and the query business domain associated with the query request includes: Obtain the access token carried in the query request, and parse the access token carried in the query request to obtain the first tenant identifier; The core business keywords are extracted from the query request and matched with a preset business domain knowledge base to determine the query business domain to which the query request belongs.

[0055] Preferably, in some embodiments of this application, upon receiving a query request, the query request is parsed into a structured search request. The structured search request includes at least a tenant identifier, query keywords, and optional data table constraint information. The structured search request may include a query business domain.

[0056] Preferably, in some embodiments of this application, the first tenant identifier can be obtained in any of the following ways: 1. directly from the caller context, such as user session, API header, authentication token, etc.; 2. if the caller does not provide it, the default tenant is obtained according to the configuration mapping of the user's organization / project and used as the first tenant identifier.

[0057] Preferably, in some embodiments of this application, the extraction of core business keywords from the query request includes: performing keyword extraction and normalization processing on the query request to obtain a keyword set for subsequent directory retrieval and field retrieval. Specific rules include: extracting noun phrases, business entity words, and measurement terms; removing stop words and meaningless words; and normalizing numbers, units, and synonyms. For example, if the query request is "fee amount," the core business keyword obtained is "fee amount."

[0058] Preferably, in some embodiments of this application, the process of determining the query business domain includes at least one of the following: if the user specifies a business domain in the interface or call parameters, the specified business domain is used as the query business domain; if the user does not specify a business domain in the interface or call parameters, the business domain tags included in the data table in the data table limitation information are used as the business domain knowledge base, and the core business keywords input by the user are matched to select the business domain tag with the highest matching degree as the query business domain; if the matching degree between each business domain tag in the current business domain knowledge base and the core business keywords does not meet the preset conditions, all business domain tags in all data tables are used as a new business domain knowledge base, and the matching process between the business domain tags and the core business keywords is further executed.

[0059] By parsing the access token carried in the query request to determine the tenant identifier, and matching the core business keywords in the query request with the business domain knowledge base, the query business domain is automatically determined, thus avoiding reliance on manually specifying the tenant or business scope. This approach enables the accurate and automatic extraction and injection of tenant information and business semantics from the query context into the metadata retrieval process, ensuring the accuracy and reliability of the preconditions for subsequent table index matching and business domain filtering. In a multi-tenant, multi-business environment, this mechanism helps reduce the risk of erroneous queries across tenants and businesses, making the metadata query process more efficient and consistent with the actual business context.

[0060] S103: Using all the candidate data tables and the query business domain as range filtering conditions, and in accordance with the order of the retrieval priority, perform metadata retrieval at each of the index levels based on the second association relationship until the target metadata that meets the preset confidence conditions is obtained, and output the target metadata.

[0061] Furthermore, in some embodiments of this application, the step of using all the candidate data tables and the query business domain as range filtering conditions, and performing metadata retrieval at each of the index levels according to the second association relationship in order of retrieval priority, includes: Based on the query request, negative semantic keywords are generated, and filtering instructions are generated based on the negative semantic keywords; When metadata retrieval is performed sequentially in each of the index levels, the filtering instructions are used to perform semantic consistency checks on each of the metadata associated in the current index level, and the metadata indexes corresponding to the metadata that match the meaning of the negative semantic keywords are removed to obtain the first index set; The first index set is filtered by all the candidate data tables and the query business domain to obtain the second index set, and metadata retrieval of the corresponding index level is performed in the second index set according to the second association relationship.

[0062] Preferably, in some embodiments of this application, generating negative semantic keywords based on the query request includes: performing semantic analysis on the query request using a preset large language model to obtain the query intent, and further generating negative semantic keywords that are semantically opposite to the query intent based on the query intent.

[0063] Preferably, in some embodiments of this application, before performing metadata retrieval based on the second association at each of the index levels, a preliminary metadata query is first performed in the static data layer. The static data layer ensures that even if a newly added data table exists in the database, and the metadata in that table has not undergone semantic enhancement and hierarchical indexing, the metadata in that data table can still be retrieved.

[0064] Preferably, in some embodiments of this application, the step of using the filtering instruction to perform semantic consistency verification on each of the metadata associated in the current index level, and removing the metadata index corresponding to the metadata that matches the meaning of the negative semantic keyword to obtain a first index set includes: using a large language model to perform semantic consistency verification between the enhanced semantic description and the negative semantic keyword stored in the metadata index, removing the metadata index that has semantic contradictions between the two, and obtaining a first index set.

[0065] Preferably, in some embodiments of this application, the step of filtering the corresponding first index set through all the candidate data tables and the query business domain to obtain the second index set includes: since the metadata index includes the table name of the data table to which the metadata belongs and the business domain label of the metadata, a batch of metadata indexes that are in the known candidate data tables and whose business domain labels are the query business domains can be initially locked in the first index set to obtain the second index set.

[0066] To address the issue that relying solely on business domain filtering may still introduce semantically irrelevant metadata, this method introduces negative semantic keywords to generate filtering instructions and performs semantic consistency checks during the retrieval process at each index level. This allows for the early removal of metadata indexes that do not conform to query semantic constraints. This approach restricts metadata retrieval not only to the table and business domain scopes but also further to the exclusive constraints of query semantics, thereby reducing the probability of invalid matches within the index level. By moving the semantic filtering stage forward in the retrieval phase, the size of candidate indexes can be effectively reduced, allowing subsequent metadata retrieval to focus more on semantically relevant objects, thus improving retrieval efficiency and accuracy in complex business semantic query scenarios.

[0067] Furthermore, in some embodiments of this application, obtaining target metadata that satisfies preset confidence conditions includes: The metadata associated with the second index set is used as the first candidate metadata. The consistency between the first candidate metadata and the query request is evaluated to obtain the semantic relevance of each first candidate metadata. When metadata retrieval is performed sequentially in each of the index levels, the level weight corresponding to the current index level is obtained. By weighting the level weight and the semantic relevance, the comprehensive confidence score of each of the first candidate metadata in the current index level is obtained. If there are several second candidate metadata whose overall confidence score is greater than a preset threshold among all the first candidate metadata in the current index level, then the second candidate metadata with the largest overall confidence score will be used as the target metadata.

[0068] Preferably, in some embodiments of this application, the semantic relevance can be estimated using methods such as Euclidean distance or cosine similarity to determine the degree of correlation between two data points. This application does not limit the calculation method of semantic relevance.

[0069] Preferably, in some embodiments of this application, the formula for calculating the comprehensive confidence score is: ; in, For metadata The overall confidence score; For metadata The score of semantic relevance to the query request; For metadata The semantic confidence level stored in the corresponding metadata index; For metadata The weight of the index level corresponding to the metadata index is as follows: the weight of the index level of the commonly used table index layer, the ordinary index layer, and the cold data index layer decreases in that order. , as well as These are the weight coefficients for the corresponding weighted terms.

[0070] Based on the second index set, a consistency assessment of candidate metadata and query requests is performed, introducing semantic relevance and combining index level weights to calculate a comprehensive confidence score, thereby achieving refined sorting and filtering of different candidate metadata. This approach ensures that the selection of target metadata no longer relies solely on a single level or a single semantic similarity, but comprehensively considers the semantic matching degree of the metadata itself and the importance of its index level. When high-confidence metadata is identified in advance in a high-priority index layer, the subsequent retrieval process can be directly terminated, avoiding invalid traversal, further compressing query response time, and improving the ability to quickly locate metadata.

[0071] In summary, the database query method provided in this application has the following advantages compared to existing technologies: This application distinguishes database metadata by business domain tags and constructs a two-level inverted index structure containing table indexes and metadata indexes based on the business domain tags, enabling metadata to be logically organized in an orderly manner according to tenant and business semantic dimensions. When receiving a query request, the range of candidate data tables is first limited in the table index using tenant identifiers, and then the scope of metadata retrieval is constrained by the query business domain, thereby avoiding indiscriminate retrieval of all database objects and significantly reducing the retrieval space. Furthermore, by introducing multi-level retrieval priorities based on access characteristics into the metadata index and executing metadata retrieval step by step according to priority order during the retrieval process, the query process can prioritize matching in highly relevant index levels. The retrieval process can be terminated early after meeting the pre-set confidence conditions, reducing unnecessary index access and computation overhead. Ultimately, this achieves targeted and efficient metadata querying in multi-tenant, multi-business coexistence scenarios, improving query response speed and overall system retrieval efficiency.

[0072] like Figure 2 As shown, based on the above method embodiments, an embodiment of this application provides a database query device, including: an index building module 201, a table filtering module 202, and a data retrieval module 203; The index building module 201 is used to determine the business domain label of each piece of metadata based on the acquired metadata of the database to be queried, and to build a two-level inverted index according to the business domain label; wherein, the two-level inverted index includes: a table index and a metadata index; the table index stores a first association relationship between the tenant identifier and the data table; the metadata index stores a second association relationship between the metadata and the business domain label, and the metadata index is divided into multiple index levels with different retrieval priorities according to the access characteristics of the corresponding metadata; The table filtering module 202 is used to receive a query request, obtain the first tenant identifier and query business domain associated with the query request, and perform matching in the table index according to the first tenant identifier and the first association relationship to determine several candidate data tables. The data retrieval module 203 is used to perform metadata retrieval at each of the index levels according to the second association relationship, based on the second association relationship, using all the candidate data tables and the query business domain as range filtering conditions, and in the order of the retrieval priority, until the target metadata that meets the preset confidence conditions is obtained, and then outputs the target metadata.

[0073] Furthermore, in some embodiments of this application, the index building module 201 includes: a statistics unit and a hierarchy division unit; the index building module 201 is used to divide the metadata index into multiple index levels with different retrieval priorities according to the access characteristics of the corresponding metadata, including: The statistical unit is used to count the number of times the metadata is retrieved within a preset period, and to assign a popularity score to the metadata based on the number of times it is retrieved. The hierarchical division unit is used to determine the retrieval priority of the metadata based on the popularity score, and to divide the corresponding metadata index into the corresponding index level according to the retrieval priority.

[0074] Furthermore, in some embodiments of this application, the index hierarchy includes: a frequently used table index layer, a regular index layer, and a cold data index layer; the hierarchy division unit is used to determine the retrieval priority of the metadata based on the popularity score, and to divide the corresponding metadata index into the corresponding index hierarchy according to the retrieval priority, including: When the number of times the metadata corresponding to the metadata index is retrieved is greater than a preset frequency, the score range to which the popularity score of the metadata index belongs is determined. If the popularity score is greater than the first preset score, the metadata index will be assigned to the commonly used table index layer with the first retrieval priority. If the popularity score is greater than the second preset score but not greater than the first preset score, then the metadata index is assigned to the ordinary index layer with the second retrieval priority. If the popularity score is not greater than the second preset score, the metadata index will be assigned to the cold data index layer with the third retrieval priority.

[0075] Further, in some embodiments of this application, the data retrieval module 203 includes: an instruction generation unit, a first filtering unit, and a second filtering unit; the data retrieval module 203 is used to perform metadata retrieval at each of the index levels according to the second association relationship, using all the candidate data tables and the query business domain as range filtering conditions, and in accordance with the retrieval priority order, including: The instruction generation unit is used to generate negative semantic keywords based on the query request, and to generate filtering instructions based on the negative semantic keywords; The first filtering unit is used to perform semantic consistency verification on each of the metadata associated in the current index level by using the filtering instruction when performing metadata retrieval in each of the index levels in sequence, and to remove the metadata index corresponding to the metadata that matches the meaning of the negative semantic keyword, so as to obtain the first index set. The second filtering unit is used to filter the corresponding first index set through all the candidate data tables and the query business domain to obtain a second index set, and to perform metadata retrieval of the corresponding index level in the second index set according to the second association relationship.

[0076] Furthermore, in some embodiments of this application, the data retrieval module 203 includes: a confidence assessment unit, a weighting unit, and a determination unit; the data retrieval module 203 is used to obtain target metadata that meets preset confidence conditions, including: The confidence assessment unit is used to take the metadata associated with the second index set as the first candidate metadata, perform a consistency assessment between the first candidate metadata and the query request, and obtain the semantic relevance of each first candidate metadata. The weighting unit is used to obtain the level weight corresponding to the current index level when performing metadata retrieval in each index level in sequence, and to obtain the comprehensive confidence score of each first candidate metadata in the current index level by weighting the level weight and the semantic relevance. The determination unit is configured to, if among all the first candidate metadata in the current index level there are several second candidate metadata whose comprehensive confidence score is greater than a preset threshold, then take the second candidate metadata with the largest comprehensive confidence score as the target metadata.

[0077] Furthermore, in some embodiments of this application, the index building module 201 includes: a third filtering unit, a semantic enhancement unit, and a classification unit; the index building module 201 is used to determine the business domain label of each piece of metadata based on the acquired metadata of the database to be queried, including: The third filtering unit is used to remove the first metadata that does not have business semantics from all the metadata using preset filtering rules, so as to obtain the third candidate metadata. The semantic enhancement unit is used to generate an enhanced semantic description of the third candidate metadata and several corresponding distinguishing data with the same semantics through a preset large language model. The classification unit is used to obtain the context information of the third candidate metadata in the database to be queried, and determine the business domain label of each third candidate metadata according to the context information, the enhanced semantic description and all the distinguishing data.

[0078] Further, in some embodiments of this application, the table filtering module 202 includes: a tenant confirmation unit and a matching unit; the table filtering module 202 is used to receive a query request, obtain a first tenant identifier associated with the query request and a query business domain, including: The tenant confirmation unit is used to obtain the access token carried in the query request and parse the access token carried in the query request to obtain the first tenant identifier. The matching unit is used to extract core business keywords from the query request, and match the core business keywords with a preset business domain knowledge base to determine the query business domain to which the query request belongs.

[0079] In summary, the database query device provided in this application has the following advantages compared to the prior art: This application distinguishes database metadata by business domain tags and constructs a two-level inverted index structure containing table indexes and metadata indexes based on the business domain tags, so that the metadata is logically organized in an orderly manner according to the tenant dimension and business semantic dimension. When receiving a query request, the range of candidate data tables is first limited in the table index using the tenant identifier, and then the range of metadata retrieval is constrained by the query business domain, thereby avoiding indiscriminate retrieval of all database objects and significantly reducing the retrieval space. Furthermore, by introducing multi-level retrieval priorities divided by access characteristics in the metadata index, and executing metadata retrieval level by level according to priority order during the retrieval process, the query process can first complete the matching in the highly relevant index level. The retrieval process can be terminated in advance after the pre-set confidence condition is met, reducing unnecessary index access and calculation overhead. Finally, it realizes targeted and efficient query of metadata in multi-tenant and multi-business coexistence scenarios, improving query response speed and overall system retrieval efficiency.

[0080] It is understood that the above-described device embodiments correspond to the method embodiments of this application, and can implement the database query method provided by any of the above-described method embodiments of this application.

[0081] It should be noted that the device embodiments described above are merely illustrative, and some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can specifically be implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0082] Based on the above-described embodiments of the database query method, another embodiment of this application provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the database query method of any embodiment of this application.

[0083] For example, in this embodiment, the computer program can be divided into one or more modules, which are stored in the memory and executed by the processor to complete this application. The one or more module units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0084] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0085] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0086] Based on the above-described method embodiments, another embodiment of this application provides a computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to execute the database query method described in any of the above-described method embodiments of this application.

[0087] The modules / units integrated in the device / terminal equipment, if implemented as software functional units and sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

Claims

1. A database query method, characterized in that, include: Based on the acquired metadata of the database to be queried, a business domain tag for each piece of metadata is determined, and a two-level inverted index is constructed according to the business domain tag. The two-level inverted index includes a table index and a metadata index. The table index stores a first association between the tenant identifier and the data table. The metadata index stores a second association between the metadata and the business domain tag, and the metadata index is divided into multiple index levels with different retrieval priorities according to the access characteristics of the corresponding metadata. Receive a query request, obtain the first tenant identifier and query business domain associated with the query request, and match the table index according to the first tenant identifier and the first association relationship to determine several candidate data tables; Using all the candidate data tables and the query business domain as range filtering conditions, metadata retrieval is performed at each of the index levels according to the second association relationship in the order of the retrieval priority until the target metadata that meets the preset confidence conditions is obtained, and the target metadata is output.

2. The database query method as described in claim 1, characterized in that, The metadata index is divided into multiple index levels with different retrieval priorities based on the access characteristics of the corresponding metadata, including: The number of times the metadata is retrieved within a preset period is counted, and the popularity score of the metadata is determined based on the number of times it is retrieved. The retrieval priority of the metadata is determined based on the popularity score, and the corresponding metadata index is divided into the corresponding index level according to the retrieval priority.

3. The database query method as described in claim 2, characterized in that, The index hierarchy includes: a frequently used table index layer, a regular index layer, and a cold data index layer; the step of determining the retrieval priority of the metadata based on the popularity score, and dividing the corresponding metadata index into the corresponding index hierarchy according to the retrieval priority, includes: When the number of times the metadata corresponding to the metadata index is retrieved is greater than a preset frequency, the score range to which the popularity score of the metadata index belongs is determined. If the popularity score is greater than the first preset score, the metadata index will be assigned to the commonly used table index layer with the first retrieval priority. If the popularity score is greater than the second preset score but not greater than the first preset score, then the metadata index will be assigned to the ordinary index layer with the second retrieval priority. If the popularity score is not greater than the second preset score, the metadata index will be assigned to the cold data index layer with the third retrieval priority.

4. The database query method as described in claim 1, characterized in that, The step of using all the candidate data tables and the query business domain as range filtering conditions, and performing metadata retrieval at each of the index levels according to the second association relationship in order of retrieval priority, includes: Based on the query request, negative semantic keywords are generated, and filtering instructions are generated based on the negative semantic keywords; When metadata retrieval is performed sequentially in each of the index levels, the filtering instructions are used to perform semantic consistency checks on each of the metadata associated in the current index level, and the metadata indexes corresponding to the metadata that match the meaning of the negative semantic keywords are removed to obtain the first index set; The first index set is filtered by all the candidate data tables and the query business domain to obtain the second index set, and metadata retrieval of the corresponding index level is performed in the second index set according to the second association relationship.

5. A database query method as described in claim 4, characterized in that, The acquisition of target metadata that meets the preset confidence conditions includes: The metadata associated with the second index set is used as the first candidate metadata. The consistency between the first candidate metadata and the query request is evaluated to obtain the semantic relevance of each first candidate metadata. When metadata retrieval is performed sequentially in each of the index levels, the level weight corresponding to the current index level is obtained. By weighting the level weight and the semantic relevance, the comprehensive confidence score of each of the first candidate metadata in the current index level is obtained. If there are several second candidate metadata whose overall confidence score is greater than a preset threshold among all the first candidate metadata in the current index level, then the second candidate metadata with the largest overall confidence score will be used as the target metadata.

6. The database query method as described in claim 1, characterized in that, The process of determining the business domain label for each piece of metadata based on the acquired metadata of the database to be queried includes: By using preset filtering rules, the first metadata that does not have business semantics is removed from all the metadata to obtain the third candidate metadata; An enhanced semantic description of the third candidate metadata and several corresponding distinguishing data with the same semantics are generated using a pre-defined large language model. Obtain the context information of the third candidate metadata in the database to be queried, and determine the business domain label of each third candidate metadata based on the context information, the enhanced semantic description, and all the distinguishing data.

7. A database query method as described in any one of claims 1 to 6, characterized in that, The step of receiving a query request and obtaining the first tenant identifier and query business domain associated with the query request includes: Obtain the access token carried in the query request, and parse the access token carried in the query request to obtain the first tenant identifier; The core business keywords are extracted from the query request and matched with a preset business domain knowledge base to determine the query business domain to which the query request belongs.

8. A database query device, characterized in that, include: The module includes an index building module, a table filtering module, and a data retrieval module. The index building module is used to determine the business domain label of each piece of metadata based on the acquired metadata of the database to be queried, and to build a two-level inverted index based on the business domain label; wherein, the two-level inverted index includes: a table index and a metadata index; the table index stores a first association relationship between the tenant identifier and the data table; the metadata index stores a second association relationship between the metadata and the business domain label, and the metadata index is divided into multiple index levels with different retrieval priorities according to the access characteristics of the corresponding metadata; The table filtering module is used to receive a query request, obtain the first tenant identifier and query business domain associated with the query request, and match the table index according to the first tenant identifier and the first association relationship to determine several candidate data tables. The data retrieval module is used to perform metadata retrieval at each of the index levels according to the second association relationship, based on the second association relationship, using all the candidate data tables and the query business domain as range filtering conditions, and in the order of the retrieval priority, until the target metadata that meets the preset confidence conditions is obtained, and then outputs the target metadata.

9. A terminal device, characterized in that, The system includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements a database query method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform a database query method as described in any one of claims 1 to 7.

Images